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ABSTRACT 



A printed object, such as an item of postal mail, a book, 
printed advertising, a business card, product packaging, etc., 
is steganographically encoded with plural-bit data. When 
such an object is presented to an optical sensor, the plural-bit 
data is decoded and used to establish a link to an internet 
address corresponding to that object. 

23 Claims, 2 Drawing Stieets 
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LINKING OF COMPUTERS BASED ON 
OPTICAL SENSING OF DIGITAL DATA 

RELATED APPLICATION DATA 

This application is a continuation-in-part of copending ^ 
application Ser. No. 09/130,624, filed Aug. 6, 1998, which 
is a continuation of application Set. No. 08/508,083 filed on 
Jul. 27, 1995, (now U.S. Pat. No. 5,841,978). This applica- 
tion is also a continuation-in part of copending application 
Ser. No. 09/314,648, filed May 19, 1999 (attached as >^pen- 
dix A). This application is also a continuation-in-part of 
copending provisional application 60/134,782, also filed 
May 19, 1999 (attached as Appendix B). This application is 
also a continuation-in-part of copending application Ser. No. 
09/292,569, filed Apr. 15, 1999, which claims priority to 
application Ser. No. 60/082,228, filed Apr. 16, 1998. 

HELD OF THE INVENTION 

The present invention relates optical user interfaces that 20 
sense digitally-encoded objects. The invention further 
relates to systems using such optical interfaces to control 
computers, and to navigate over or act as portals on net- 
works. 

25 

BACKGROUND AND SUMMARY OF THE 
INVENTION 

"Bedoop." That might be the sound that someone might 
hear as they lazily place a magazine advertisement in front 
of their desktop camera. Magically, the marketing and sales 
web site associated with the ad is displayed on their com- 
puter. More information? Want to buy now? Look at the full 
product fine? No problem. 

"Bedoop." That might be the same soimd when that same 
someone places their credit card in front of their desktop 
camera. Instantly, the product displayed on the web page is 
purchased. Behind the scenes, a secure purchase link is 
initiated, transmitting all requisite information to the vendor. 
Twist the credit card clockwise and the purchaser chooses 
overnight dehvery. 

So goes an exemplary embodiment of the invention 
fiu-ther described in this application. Though this example is 
rather specific, it nevertheless alludes to an indescribably 
vast array of applications possible when a digital camera or 45 
other optical sensing device is turned into a general purpose 
user interface device with an intuitive power that very well 
might rival the mouse and the keyboard. 

The centerpiece of the invention is that an object or paper 
product so-scanned contains digital information that can be 50 
quickly read and acted upon by an appropriately configured 
device, computer or appliance. The preferred embodiment 
envisions that this digital information is aesthetically hidden 
on objects. These objects have been previously and pro- 
actively marked with the digital information, using any of 55 
the broad ranges of printing and processing techniques 
which are available on the market and which are widely 
described in the open literature and patent hterature sur- 
rounding digital watermarking. 

Be this as it may, though the invention concentrates on flat 60 
object applications wherein the digital infonnation is often 
imperceptibly integrated into the object, it is certainly not 
meant to be so limited. Objects can be three dimensional in 
nature and the information more visually overt and/or pre- 
existing (i.e., not "pro-actively" embedded, or not even be 65 
"digital," per se). DilTerenl implementation considerations 
attach to these variants. Likewise, though the bulk of this 
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disclosure concentrates on objects which have some form of 
digital message attached thereto, some aspects of the inven- 
tion may apply to objects which have no such thing, where 
the prior arts of pattern recognition and gestural input can be 
borrowed in combination with this invention to effect yet a 
broader array of applications. 

"Bedoop." The sound that a refrigerator might make, 
outfitted with a simple camera/processor unit/net 
connection, as the ten year old holds up the empty milk 
carton and a ping goes out to the local grocery store, adding 
the item to an accumulating delivery list. The sound that 
might be heard echoing over and over inside Internet cafes 
as heretofore computerphobes take their first skeptical steps 
onto the world wide web. The sound heard at the fast food 
counter as the repeat customer holds up their sandwich card 
ticking off their latest meal, hoping for the sirens to go off 
for a $500 prize given to the lucky customer of the week. 
Blue sky scenarios abound. 

This invention is therefore about powerful new tiser 
interfaces to computers involving optical input. These new 
user interfaces extend into the everyday world in ways that 
a mouse and keyboard never could. By enabling everyday 
objects to communicate their identities and functions to 
ever-attendant devices, not only will the world wide web be 
given an entirely new dimension, but basic home and oflBce 
computing may be in store for some fundamental advances 
as well. 

These and a great many other features of the present 
invention will be more readily apparent from the follov^ng 
detailed description, which proceeds with reference to the 
accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram showing one embodiment of the 
present invention. 

FIG. 2 is another block diagram showing an embodiment 
of the present invention. 

DETAILED DESCRIPTION 

Basically, the technology detailed in this disclosure may 
be regarded as enhanced systems by which users can interact 
with computer-based devices. Their simple nature, and 
adaptability for use with everyday objects (e.g., milk 
cartons), makes the disclosed technology well suited for 
countless applications. 

Due to the great range and variety of subject matter 
detailed in this disclosure, an orderly presentation is difficult 
to achieve. As will be evident, many of the topical sections 
presented below are both founded on, and foundational to, 
other sections. For want of a better rationale, the sections are 
presented below in a more or less random order. It should be 
recognized that both the general principles and the particular 
details from each section find application in other sections as 
well. To prevent the length of this disclosure from balloon- 
ing out of control, the various permutations and combina- 
tions of the features of the different sections are not exhaus- 
tively detailed. The inventors intend to explicitly teach such 
combinations/permutations, but practicality requires that the 
detailed synthesis be left to those who ultimately implement 
systems in accordance with such teachings. 
Basic Principles — Refrigerators and Clutter 

Referring to FIG. 1, a basic embodiment 10 of the present 
invention includes an optical sensor 12, a computer 14, and 
a network connection 16 to the internet 18, The illustrated 
optical sensor 12 is a digital camera having a resolution of 
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320 by 200 pixels (color or black and white) that stares out, 
grabbing frames of image data five times per second and 
storing same in one or more frame buffers. These frames of 
image data are analyzed by a computer 14 for the presence 
of Bedoop data. (Essentially, Bedoop data is any form of 5 
digital data encoding recognized by the system 10— data 
which, in many embodiments, initiates some action.) Once 
detected, the system responds in accordance with the 
detected Bedoop data (e.g., by initiating some local action, 
or by communication with a remote computer, such as over 
the internet, via an online service such as AOL, or using 
point-to-point dial-up communications, as with a bulletin 
board system. 

Consider the milk carton example. The artwork on a milk 
carton can be adapted to convey Bedoop data. In the 
preferred embodiment, the Bedoop data is steganographi- 
cally encoded (e.g., digitally watermarked) on the carton. 
Numerous digital watermarking techniques are known — all 
of which convey data in a hidden form (i.e., on human 
inspection, it is not apparent that digitally encoded data is 
present). Exemplary techniques operate by slightly changing 20 
the luminance, or contours, of selected points on artwork or 
text printed on the carton, or splatter tiny droplets of ink on 
the carton in a seemingly random pattern. Each of these 
techniques has the effect of changing the local luminance at 
areas across the carton — ^luminance changes that can be 25 
detected by the computer 14 and decoded to extract the 
encoded digital data. In the case of a milk carton, the data 
may serve to identify the object as, e.g., a half gallon carton 
of Alpenrose brand skim milk. 

The FIG. 1 apparatus can be integrated into the door of a 
refrigerator and used to compile a shopping list. Milk 
cartons, and other Bedoop-encoded packaging 20, can be 
held up the optical sensor. When the computer 14 detects the 
presence of Bedoop data and successfully decodes same, it 
issues a confirmation tone ("be-doop") from a speaker or 
other audio transducer 22. The computer then adds data ^5 
identifying the just -detected object to a grocery list. This list 
can be maintained locally (in disk storage, non-volatile 
RAM 24, or the like in the refrigerator, or elsewhere in the 
home), or remotely (e.g., at a server located at a user- 
selected grocery, or elsewhere). In either event, the list is 40 
desirably displayed on a display in the user's home (e.g., an 
LCD screen 26 built into the front of the appliance). Con- 
ventional user interface techniques can be employed per- 
mitting the user to scroll through the displayed list, delete 
items as desired, etc. 45 

Periodically, the listed groceries can be purchased and the 
list cleared. In one embodiment, the list is printed (either at 
the home or at the grocery), and the user walks the grocery 
aisles and purchases same in the conventional manner. In 
another embodiment, the grocer pulls the listed items from 50 
the shelves (in response to a user request conveyed by the 
internet or telephone, or by a gestuire as hereafter detailed). 
Once the list has been pulled, the grocer can alert the user 
that the groceries are available for pickup (again, e.g., by 
internet or telephone message), or the grocer can simply 55 
deliver the groceries directly to the user's home. Naturally, 
on-line payment mechanisms can be employed if desired. 

Consider a wholly unrelated Bedoop application. An 
Excel spreadsheet is printed onto paper, and the paper 
becomes buried in a stack of clutter on an ofSce worker's 60 
desk. Months later the spreadsheet again becomes relevant 
and is dug out of the stack. Changes need to be made to the 
data, but the file name has long-since been forgotten. The 
worker simply holds the dug-out page in front of a camera 
associated with the desktop computer. A moment later, the 65 
electronic version of the file appears on the worker's com- 
puter display. 
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When the page was originally printed, tiny droplets of ink 
or toner were distributed across the paper in a pattern so light 
as to be essentially un-noticeable, but which steganographi- 
cally encoded the page with a plural-bit binary number (e.g., 
64 bits). A database (e.g., maintained by the operating 
system, the Excel program, the printer driver, etc.) stored 
part of this number (e.g., 24 bits, termed a Univeral Identifier 
or UID) in association with the path and file name at which 
the electronic version of the file was stored, the page number 
within the document, and other useful information (e.g., 
author of the file, creation date, etc.). 

The steganographic encoding of the document, and the 
updating of the database, can be performed by the software 
application (e.g.. Excel). This option can be selected once by 
the user and applied thereafter to all printed documents (e.g., 
by a user selection on an "Options" drop-down menu), or 
can be presented to the user as part of the Print dialog 
window and selected (or not) for each print job. 

When such a printed page is later presented to the camera, 
the computer automatically detects the presence of the 
encoded data on the page, decodes same, consults the 
database to identify the file name/location/page correspond- 
ing to the UID data, and opens the identified file to the 
correct page (e.g., after launching Excel). This application is 
one of many "paper as portal" applications of the Bedoop 
technology. 

The foregoing are but two of myriad applications of the 
technology detailed herein. In the following discussion a 
great many other applications are disclosed (some 
groundbreaking, a few gimmicky). However, regardless of 
the length of the specification, it is possible only to begin to 
explore a few of the vast ramifications of this technology. 

A few more details on the basic embodiments described 
above may be helpful before delving into other applications. 
Optics 

For any system to decode steganographically encoded 
data from an object, the image of the object must be 
adequately focused on the digital camera's CCD (or other) 
sensor. In a low cost embodiment, the camera has a fixed 
nominal focal length, e.g., in the range of 6-24 inches 
(greater or lesser lengths can of course be used). Since the 
camera is continuously grabbing and analyzing frames of 
data, the user can move the object towards- or away- from 
the sensor until the decoder succeeds in decoding the 
steganographically encoded data and issues a confirming 
"Bedoop" audio signal. 

In more elaborate embodiments, known auto-focusing 
technology can be employed. 

In still other embodiments, the camera (or other sensor) 
can be equipped with one or more auxiliary fixed-focus 
lenses that can be selectively used, depending on the par- 
ticular application. Some such embodiments have a first 
fixed focused lens that always overlies the sensor, with 
which one or more auxiliary lenses can be optically cas- 
caded (e.g., by hinge or slide arrangements). Such arrange- 
ments are desirable, e.g., when a camera is not a dedicated 
Bedoop sensor but also performs other imaging tasks. When 
the camera is to be used for Bedoop, the auxiliary lens is 
positioned (e.g., flipped into) place, changing die focal 
length of the first lens (which may by unsuitably long for 
Bedoop purposes, such as infinity) to an appropriate Bedoop 
imaging range (such as one foot). 

Other lens-switching embodiments do not employ a fixed 
lens that always overlies the sensor, but instead employ two 
or more lenses that can be moved into place over the sensor. 
By selecting different lenses, focal lengths such as infinity, 
six feet, and one foot can be selected. 
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In all such arrangements, it is desirable (but not essential) fiducials), or serving another purpose as well (e.g. lines of 

that the steganographically-encoded portion of the object text), to discern orientation. Edge-detection algorithms can 

being imaged fills a substantial part of the image frame. The also be employed to deduce the orientation of the object by 

object can be of various sizes, e.g., an 10 by 12 inch front reference to its edges. 

panel of a cereal box, or a proof of purchase certificate that 5 Some embodiments filter the image data at some point m 

is just one inch square. To meet this requirement, small t^e process to aid in ultimate Bedoop data extracUon One 

objects will obviously need to be placed closer to the camera ^ such filtering is to mitigate image data artifacts due to 

than large objects. The optics of the system can be designed, ^he particular optical sensor. For example, CCD arrays have 

e.g., by selection of s Jable aperture sizing and auxiUary regdarly-spaced sensors that sampk image at 

1- u*- /-r J -i\ * 1 • u- ♦ . uniformly spaced discrete points. This discrete samphng 

hghtmg (If needed), to properly image objects of vanous lO ^^^^^ ^ f^ai^formation of the image data, leading to certain 

sizes withm a range of focal distances. ■ artifacts. An appropriately configured filter can miti- 

Some embodiments avoid issues of focal distances and gate the effect of these artifacts, 

identifying the intended object by constraining the size of ^j, arrangements, the step of determining the ori- 

the object and/or its placement. An example is a business entation can be omitted. Business card readers, for example, 

card reader that is designed for the sole task of imaging is produce data that is reliably free of artifacts and is of known 

business cards. Various such devices are known. scale. Or the encoding of the Bedoop data can be effected in 

Decoding/Encoding such a way that renders it relatively immune to certain 

The analysis of the image data can be accomplished in distortion mechanisms. For example, while the presently- 
various known ways. Presently, most steganographic decod- preferred encoding arrangement operates on a 2D grid basis, 
ing relies on general purpose microprocessors that are 20 with rows and columns of data points, the encoding can 
programmed by suitable software instructions to perform the alternatively be done on another basis (e.g., a rotationally- 
necessary analysis. Other arrangements, such as using dedi- symmetric form of encoding, such as a 2D bar-code, so that 
cated hardware, reprogrammable gate arrays, or other rotational state of the image data can be ignored). In still 
techniques, can of course be used. other embodiments, the orientation-determining step can be 

The steganographic decoding process may entail three 25 omitted because the decoding can readily proceed without 

steps. In the first, the object is located. In the second, the this information. For example decoding which relies on the 

object's orientation is discemed. In the third, the Bedoop Fourier-Mellin transform produces data in which scale and 

data is extracted from the image data corresponding to the rotation can be ignored.) 

Bedoop object. Once the orientation of the object is discemed, the image 
The first step, object location, can be assisted by various 30 data may be virtually re-registered, effectively mapping it to 
clues. One is the placement of the object; typically the center another perspective (e.g., onto a rectilinear image plane), 
of the image field will be a point on the object. The This mapping can employ known image processing tech- 
surrounding data can then be analyzed to try and discern the niques to compensate, e.g., for rotation state, scale state, 
object's boundaries. differential scale state, and X-Y offset, of the original 
Another location technique is slight movement. Although 35 Bedoop image data. The resulting frame of data may then be 
the user will typically try to hold the object still, there will more readily processed to extract the steganographically- 
usually be some jitter of the Bedoop object within the image encoded Bedoop data. 

frame (e.g., a few pixels back and forth). Background visual In the preferred embodiment, after the image data is 

clutter, in contrast, will typically be stationary. Such move- remapped into rectilinear planar form, subliminal graticule 

ment may thus be sensed and used to identify the Bedoop 40 data is sensed that identifies the locations within the image 

object from within the image data. data where the binary data is encoded. Desirably, the binary 

Still another object-location clue is object shape. Many data is redundantly encoded, e.g., in 8x8 patch blocks. Each 

Bedoop objects are rectangular in shape (or trapezoidal as patch comprises one or more pixels. (The patches are 

viewed by the camera). Straight edge boundaries can thus be typically square, and thus contain 1, 4, 9, or 16, etc. pixels.) 

used to define an area of likely Bedoop data. 45 The nominal luminance of each patch before encoding (e.g., 

Cblor is a frirther object identification clue that may be artwork pre-existing on the object) is slightly increased or 

useful in some contexts. decreased to encode a binary "1" or "0." The change is slight 

Yet another object location clue is spatial frequency. In enough to be generally imperceptible to human observers, 

imaging systems with well defined focal zones, undesired yet statistically detectable from the image data — especially 

visual clutter may be at focal distances that results in 50 if several such blocks are available for analysis. Preferably, 

blurring. The Bedoop object, in contrast, will be in focus and the degree of change is adapted to the character of the 

may be characterized by fine detail. Analyzing the image underlying image, with relatively greater changes being 

data for the high frequencies associated with fine detail can made in regions where the human eye is less likely to notice 

be used to distinguish the intended object from others. them. Each block thus encoded can convey 64 bits of data. 

Characteristic markings on the object (as discussed below 55 The encoding of such blocks in tiled fashion across the 

in connection with determining object orientation), can also object permits the data to be conveyed in robust fashion, 

be sensed and used in locating the object. Much of the time, of course, the Bedoop sensor is staring 

Once the Bedoop object has been located within the out and grabbing image frames that have no Bedoop data, 

image data, masking can be applied (if desired) to eliminate Desirably, the detection process includes one or more checks 

image data not corresponding to the intended object. 60 to assure that Bedoop data is not wrongly discemed from 

The second step in the decoding process — determining non-Bedoop image data. Various techniques can be 

orientation of the Bedoop data — can likewise be discerned employed to validate the decoded data, e.g., error detecting 

by reference to visual clues. For example, some objects codes can be included in the Bedoop payload and checked 

include subliminal graticule data, or other calibration data, to confirm correspondence with the other Bedoop payload. 

steganographically encoded with the Bedoop data to aid in 65 Likewise, the system can confirm that the same Bedoop data 

determining orientation. Others can employ overt markings, is present in different tiled excerpts within the image data, 

either placed for that sole purpose (e.g. reference lines or etc. 
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(Details of the preferred enccxJing techniques are further 
detailed in co-pending applications Sen No. 09/293,601, 
filed Apr. 15. 1999, entitled METHODS AND DEVICES 
FOR RECOGNIZING BANKNOTES AND RESPOND- 
ING ACCORDINGLY. Ser. No. 09/127,502, filed Jul 31, 5 
1998, and U.S. Pat. No. 5,862.260.) 
Data Structures. Formats, Protocols, and Infirastructures 

In an exemplary system, the Bedoop data payload is 64 
bits. This payload is divided into three fields CLASS (12 
bits), DNS (24 bits) and UID (24 bits). (Other payload lo 
lengths, fields, and divisions, are of course possible, as is the 
provision of error-checking or error-correcting bits.) 

Within the above-described eight patch-by-eight patch 
data block, the bits are ordered row by row, starling with the 
upper left patch. The first 12 bits are the CLASS ID, 15 
followed by 24 bits of DNS data followed by 24 bits of UID 
data. (In other embodiments, the placement of bits compris- 
ing these three fields can be scrambled throughout the 
block.) 

Briefly, the CLASS ID is the most rudimentary division of 20 
Bedoop data, and may be analogized, in the familiar internet 
taxonomy, to the limited number of top level domains (e.g., 
.com, .net, .org, .mil, .edu, .jp, .de, .\ik, etc.). It is basically 
an indicator of object type. The DNS ID is an intermediate 
level of data, and may be analogized to internet server 25 
addresses (e.g., biz. yahoo, interactive.wsj, etc.). The UID is 
the finest level of granularity, and, can roughly be analogized 
to internet pages on a particular server (e.g., edition/current/ 
summaries/front. htm, daily/home/default. hts, etc.). 

Generally speaking, the CLASS ID and DNS ID, 30 
collectively, indicate to the system what sort of Bedoop data 
is on the object. In the case of Bedoop systems that rely on 
remote servers, the CLASS and DNS IDs are used in 
identifying the server computer that will respond to the 
Bedoop data The UID determines precisely what response 35 
should be provided. 

In the case of a refrigerator Bedoop system, what happens 
if an object with an unfamiUar CLASS/DNS ID data is 
encountered? The system can be programmed not to respond 
at all, or to respond with a raspberry-like sound (or other 40 
feedback) indicating "I see a Bedoop object but don't know 
what to do with it." 

Most systems wiU be able to respond to several classes of 
Bedoop objects. Simple software-based systems can com- 
pare the CLASS/DNS ID (and optionally the UID) to fixed 45 
values, and can branch program execution to corresponding 
subroutines. Likewise, hardware-based systems can activate 
different circuitry depending on the detected CLASS/DNS 
ID. 

In the case of a computer equipped with a Bedoop input 50 
device (e.g., a Sony VAIO PictureBook laptop with built-in 
camera), the operating system's registry database can be 
employed to associate different application programs with 
different CLASS/DNS IDs (just as the .XLS and .DOC file 
extensions are commonly associated by existing operating 55 
system registries to invoke Microsoft Excel and Word soft- 
ware applications, respectively). When a new Bedoop appli- 
cation is installed, it logs an entry in the registry database 
indicating the CLASS/DNS ID(s) that it will handle. 
Thereafter, when an object with such a CLASS/DNS ID is 60 
encountered, the operating system automatically launches 
the corresponding application to service the Bedoop data in 
an appropriate manner. 

Sometimes the computer system may encounter a Bedoop 
object for which it does not have a registered application 65 
program. In such case, a default Bedoop application can be 
invoked. This default application can, e.g., establish an 
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internet link to a remote server computer (or a network of 
such computers), and can transmit the Bedoop data (or a part 
of the Bedoop data) to that remote computer. The remote 
server can undertake the response itself, it can instruct the 
originating computer how to respond appropriately, or it can 
undertake some combination of these two responses. (Such 
arrangements are further considered below.) 

FIG. 2 shows an illustrative architecture employing the 
foregoing arrangement. 

At a local Bedoop system 28 (which may be implemented, 
for example, using a conventional personal computer 29), a 
camera, scanner, or other optical sensor 30 provides image 
data to a decoder 32 (which may be implemented as a 
software component of the operating system 33). The 
decoder 32 analyzes the image data to discern the plural-bit 
Bedoop data. The CLASS ID of this Bedoop data is applied 
to a Bedoop registry 34. The registry responds by identifying 
and launching a local Bedoop application 36 designed to 
service the discerned Bedoop data. 

Sometimes the system 28 may encounter a Bedoop object 
for which several different responses may be appropriate. In 
the case of a printed office docimient, for example, one 
response may be as described above — to present the elec- 
tronic version of the file on a computer, ready for editing. 
But other responses may also be desired, such as writing an 
email message to the author of the printed document, with 
the author's email address already specified in the message 
address field, etc. 

Such different responses may be handled by different 
Bedoop applications, or may be options that are both pro- 
vided by a single Bedoop application. In the former case, 
when the CLASS/DNS IDs are decoded and provided to the 
operating system, the registry indicates that there are two (or 
more) programs that might be invoked. The operating sys- 
tem can then present a dialog box to the user inviting the user 
to specify which form of response is desired. Optionally, a 
default choice can be made if the user doesn't specify within- 
a brief period (e.g., three seconds). The operating system can 
then launch the Bedoop application corresponding to the 
chosen response. 

A similar arrangement can be employed if a single 
Bedoop application can provide both responses. In such case 
the operating system launches the single Bedoop application 
(since there is no ambiguity to be resolved), and the appli- 
cation presents the choice to the user. Again, the user can 
select, or a default choice can be automatically made. 

In the just-described situations, the user can effect the 
choice by using the keyboard or mouse — as with traditional 
dialog boxes. But Bedoop provides another, usually easier, 
form of interaction. The user can make the selection through 
the optical sensor input. For example, moving the object to 
the right can cause a UI button on the right side of the dialog 
box to be selected; moving the object to the left can cause 
a UI button on the left side of the dialog box to be selected; 
moving the object towards the camera can cause the selected 
button to be activated. Many other such techniques are 
possible, as discussed below. 

If the registry 34 does not recognize, or otherwise does 
not know how to respond to Bedoop data of that particular 
CLASS/DNS, the registry launches a default Bedoop client 
application. This client application, in turn, directs a web 
browser 40 on the local Bedoop system 28 to communicate 
with a remote master registration server computer 42. The 
local computer forwards the Bedoop data to this master 
server. The master server 42 examines the CLASS ID, and 
forwards the Bedoop data (direcdy, or through intervening 
servers) to a corresponding CLASS server 44. (A single 
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server may handle Bedoop data of several classes, but more In typical rich Bedoop implementations, local systems 

typically there is a dedicated server for each CLASS.) may have Libraries of Bedoop services, applications, or 

Each CLASS server 44 serves as the root of a tree 46 of protocols. Some may be imique to that computer. Others 

distributed DNS servers. A DNS server 4Sa, for example, in may be commonly available on all computers. Some may be 

a first tier 50 of the DNS server tree, may handle Bedoop 5 highly secure, employing encryption and/or anti-hacking 

data having DNS IDs beginning with "000." Likewise, DNS measures, or data protocols that are not generally recog- 

server 4Sb may handle Bedoop data having DNS IDs nized. Others may be shareware, or the result of open-source 

beginning with "001," etc., etc. programming efforts. 

Each DNS server in the first tier 50 may, in turn, route Greeting Cards, Birthday Cards, Etc. 

Bedoop data to one of 8 servers in a second tier of the tree, lO In accordance with a further embodiment of the invention, 

in accordance with the fourth-through sixth bits of the DNS greeting cards and the like are encoded (e.g., by texturing, 

data. The tree continues in this fashion until a terminal level printing, etc.) with Bedoop data. On receiving such a card, 

of DNS leaf node servers 56. a recipient holds it in front of the image capture device on 

Ultimately, Bedoop data routed into this network reaches a laptop or other computer. The computer responds by 

aDNSleaf node server 56. That leaf node server may handle 15 displaying an internet web page that has a stock- or 

the Bedoop data, or may redirect the local Bedoop system to customized-presentation (image, video, audio-video, etc.) to 

a further server 58 that does so. That ultimate server — complement that presented on the greeting card, 

whether a DNS leaf node server or a further server — can The web site presentation can be personalized by the 

query the local Bedoop system for fiirther information, if sender (e.g., with a text message, recent family photographs, 

necessary, and can either instruct the local Bedoop system 20 etc.), either at the point of card sale, or sometime after the 

how to respond, or can undertake some or all of the response card is purchased. In the latter case, for example, the card 

itself and simply relay appropriate data back to the local can be serialized. After taking the card home, the purchaser 

Bedoop system. can visit the card vendor's web site and enter the card serial 

In arrangements in which the local Bedoop system is number in an appropriate user interface. The purchaser is 

redirected, by the DNS leaf node server, to a further server 25 then presented with a variety of simple editing tools to 

that actually handles the response, access to the further facilitate customization of the web greeting. When the 

server may be through a port 59 (e.g., a special URL) sender is finished designing the web greeting, the finished 

tailored to receipt of Bedoop data. web page data is stored (by software at the vendor's web 

In a typical implementation, most or all of the servers are site) at a site corresponding to the serial number, 

mirrored, or otherwise replicated/redundant, so that failure 30 When the card is received by a recipient and held in front 

of individual computers does not impair operation of the of a Bedoop sensor, CLASS, DNS, and UID data is decoded 

system. from the card. The CLASS and DNS data are used to 

Caching can be provided throughout the trees of servers navigate the earlier-described server network to reach a 

to speed responses. That is, responses by leaf nodes for corresponding DNS leaf node server (perhaps maintained by 

certainly commonly-encountered CLASS/DNS IDs can be 35 the Hallmark greeting card company). That leaf node server 

temporarily stored earlier in the tree(s). Bedoop data, propa- indexes a table, database, or other data structure with the 

gating through the server network, can prompt a response UID from the Bedoop data, and obtains from that data 

from an intermediate server if there is a cache hit. structure the address of an ultimate web site — the same 

If desired, Bedoop traffic through the above-detailed address at which the web greeting customized by the sender 
server trees can be monitored to collect demographic and 40 was stored. That address is provided by the DNS leaf node 
statistical information as to what systems are sending what server back to the local computer, with instructions that the 
Bedoop data, etc. One use of such information is to dynami- web page at that address be loaded and displayed (e.g., by 
cally reconfigure the DNS network to better balance server HTML redirection). The local computer complies, present- 
loads, to virtually relocate DNS resources nearer regions of ing the customized web greeting to the card recipient, 
heavy usage, etc. Another use of such information is for 45 In the just-described embodiment, in which a pre-encoded 
marketing purposes, e.g., to promote certain Bedoop fea- card is purchased by a sender and the web-display is then 
tures and applications within user groups (e.g., internet customized, the address of the web site is typicaUy deter- 
domains) that seem to under-utilize those features. mined by the card vendor. But this need not be the case. 

Within certain user networks that are linked to the Likewise, the card need not be "purchased" in the typical, 

internet, e.g., corporate networks, Bedoop data that isn't 50 card-shop fashion, 

handled within the originating Bedoop system may first be To illustrate the foregoing alternatives, consider the 
routed to a Bedoop name server within the corporate net- on-line acquisition of a greeting card, e.g., by visiting a web 
work. That server will recognize certain types of Bedoop site specializing in greeting cards. With suitable user- 
data, and know of resources within the corporate network selection (and, optionally, customization), the desired card 
suitable for handling same. Referral to such resources within 55 can be printed using an inkjet or other printer at the sender's 
the corporate network will be made, where possible. These home. In such case, the Bedoop data on the card can be 
resources (e.g., corporate servers) may respond to Bedoop similarly customized. Instead of leading to a site determined 
data in a way customized to the corporate preferences. If the by the card vendor, the data can lead to the sender's personal 
corporate Bedoop name server does not know of a resource web page, or to another arbitrary web address, 
within the corporate network that can respond to the Bedoop 60 To effect such an arrangement, the sender mtist arrange 
data, the corporate name server then routes the data to the for a DNS leaf node server to respond to a particular set of 
public Bedoop network described above. (Such referral can Bedoop data by pointing to the desired web page. While 
be to the master regisUation server or, to the extent the individuals typically will not own DNS servers, internet 
corporate name server knows the addresses of appropriate service providers commonly will Just as AOL provides 
servers within the DNS server tree, or of the further servers 65 simple tools permitting its subscribers to manage their own 
to which DNS servers may point for certain Bedoop data, it modest web pages, internet service providers can likewise 
can redirect the local Bedoop system accordingly.) provide simple tools permitting subscribers to make use of 
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DNS leaf node servers. Each subscriber may be assigned up The candidate incorporates this photo into an access 

to 20 UIDs (under a particular CLASS and DNS). The tools badge. Using a software application (which may be provided 

would permit the users to define a corresponding web especially for such purposes, e.g., as part of an ofiBce 

address for each UID. Whenever a Bedoop application led to productivity suite), the photo is dragged into an access badge 

that DNS leaf node server, and presented one of those UIDs, 5 template. The access code emailed from the employer is also 

the server would instruct the originating computer to load provided to this application. On selecting "Print," an ink-jet 

and present the web page at the corresponding web address printer associated with the candidate's computer prints out 

an access badge that includes her DMV photo and her name, 

Prior to customizing the greetmg card, the sender uses the ^ steganographicaUy encoded in accordance with 

tool provided by the internet service provider to store the ^j^^ employer-provided access code, 

address of a desired destmation web address m The name printed on the badge is obtained (by the 

dence with one of the sender s available UIDs. When j ^ * . * \ f „ #1, ^\a\/> nKTc *„ 

. . , . ^ candidate s computer) from the DMV's DNS server, m 

customizing the greeting card, the sender specines the ^ * . jr .1. u * i_ /t 

Bedoop data that is to be encoded, including the just- response to Bedoop data extracted from the photograph. (In 

referenced UID. The greeting card application encodes this ^^is application, unlike most, the photograph is not scanned 

data into the artwork and prints the resulting card. When this 15 as part of a Bedoop process. Instead, the photograph is 

card is later presented to a Bedoop system by the recipient, akeady available m digital form, so the Bedoop decoding 

the recipient's system loads and displays the web page proceeds directly from the digital representation.) 

specified by the sender. Por security purposes, the access code is not embedded 

Commerce in Bedoop Resources using standard Bedoop techniques. Instead, a non-standard 

In the just-described arrangement, internet service pro- 20 format (typically steganographic) is employed. The embed- 

viders make available to each subscriber a limited number of ding of this access code can span the entire face of the card, 

UIDs on a DNS server maintained by the service. Business or can be limited to certain regions (e.g., excluding the 

enterprises typically need greater Bedoop resources, such as region occupied by the photograph), 

their own DNS IDs (or even their own CLASS ID(s). On the appointed day the candidate presents herself at the 

While variants of the Bedoop system are extensible to 25 employer's building. At the exterior door lock, the candidate 
provide an essentially unlimited number of CLASS IDs and presents the badge to an optical sensor device, which reads 
DNS IDs, in the illustrated system these resources are the embedded building access code, checks it for authentic- 
limited. Public service, non-profit, and academic applica- ity and, if the candidate arrived within the permitted hours, 
tions should have relatively generous access to Bedoop unlocks the door. 

resources, either without charge or for only a modest charge. 30 Inside the building the candidate may encounter a security 

Business enterprises, in contrast, would be expected to pay guard. Seeing an unfamiliar person, the guard may visually 

fees to moderate their potentially insatiable demand for the compare the photo on the badge with the candidate's face, 

resources. Small businesses could lease blocks of UIDs Additionally, the guard can present the badge to a portable 

under a given CLASS/DNS ID. Larger businesses could Bedoop device, or to one of many Bedoop systems scattered 

acquire rights to entire DNS IDs, or to entire CLASS IDs (at 35 through the building (e.g., at every telephone). The Bedoop 

commensurately greater fees). system extracts the Bedoop data from the card (i.e., from the 

Web-based systems for assigning DNS IDs (and CLASS DMV photograph), interrogates the DMV's DNS server 

IDs) can be modeled after those successfiilly used by with this Bedoop data, and receives in reply the name of the 

Intemic.com, and now Networksolutions.com, for registra- person depicted in the photograph. (If the Bedoop system is 

tion of internet domains. The user fills out a web-based form 40 a telephone, the name may be displayed on a small LCD 

with names, addresses, and billing information; the system display commonly provided on telephones.) 

makes the necessary changes to all of the hidden system The guard checks the name returned by the Bedoop 

infrastructure — updating databases, routing tables, etc., in system with the name printed on the badge. On seeing that 

servers around the world, the printed and Bedoop-decoded names match (and option- 

Controlled-Access ID 45 ally checking the door log to see that a person of that name 

Just as the above-described embodiment employed an was authorized to enter and did so), the security guard can 

ink-jet printer to produce a customized-Bedoop greeting let the candidate pass. 

card, the same principles can likewise be applied to access- It will be recognized that the just-described arrangement 

control objects, such as photo-IDs. oflfers very high security, yet this security is achieved 

Consider an employment candidate who will be inter- 50 without the candidate ever previously visiting the employer, 

viewing at a new employer. The candidate's visit is without the employer knowing what the candidate looks 

expected, but she is not recognized by the building's security like, and by use of an access badge produced by the 

personnel. In this, and many other applications, arrange- candidate herself. 

ments like the following can be used: Variants of such home-printed badge embodiments find 

The employer e-mails or otherwise sends the candidate an 55 numerous applications. Consider purchasing movie- or 

access code. (The code can be encrypted for transmission.) event-tickets over the web. The user can print an access 

The code is valid only for a certain time period on a given ticket that has an entry code embedded therein. On arriving 

date (e.g., 9:00 a.m. -11:00 a.m. on Jun. 29, 1999). at the theater or event, the user presents the ticket to an 

Upon receipt of the access code, the candidate downloads optical scanning device, which decodes the entry code, 

from the web site of the state Department of Motor Vehicles 60 checks the validity of same, authorizes the entry, and marks 

the latest copy of her driver's license photo. The DMV has that entry code as having been used (preventing multiple 

already encoded this photo with Bedoop data. This data uses of tickets printed with the same code), 

leads to a state-nm DNS leaf node server 56. When that Ink- Jet Printing 

server is presented with a UID decoded from a photograph, In the foregoing discussions, reference has been made to 

the server accesses a database and returns to the inquiring 65 use of ink-jet printing as a means for providing stegano- 

computer a text string indicating the name of the person graphically encoded indicia on substrates. The following 

depicted by the photograph. discussion expands on some of the operative principles. 
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The basic physics and very low level analog electronic 
operation of ink-jet printers (sometimes termed bubble-jet 
printers) are ideally suited to support very-light-tint back- 
ground digital watermarking on any form of substrate. 
(Watermarking through apparent "tinting" of substrates is 
discussed in copending application Ser. No. 09/127^02.) In 
general, the statement, "if you can print it with an ink jet 
printer, you can watermark it" is largely accurate, even for 
(perhaps especially for) simple text documents. Indeed, 
there is a degree of flexibility and control in the ink-jet 
printing realm that is not as generally available in more 
traditional printing technologies, such as commercial offset 
printing and other plate-based technologies. (This is not to 
say that ink-jet has better quality than plate-based technolo- 
gies; it has more to do with the statistics of ink droplets than 
anything else.) Heavier tint backgrounds are possible as 
well, where the continuum ranges from very light back- 
ground tinting, where the casual observer will see "white 
paper," all the way through heavily inked patterned 
backgroimds, and photographs themselves, and everything 
in between. 

In some embodiments, the ink-jet driver software is 
modified to provide lower-level control of individual droplet 
emission than is provided in existing printer drivers, which 
are natm^ally optimized for text and graphics. In some such 
embodiments, the "watermarking" print mode is another 
option from which the user can select (e.g., in addition to 
High Quality, Econo-Fast, etc.), or the selection can be made 
automatically by application software that is printing water- 
marked data. 

In more sophisticated embodiments, the watermark data is 
applied to the printer driver software independently of the 
other image/text data. The printer driver is arranged to eject 
droplets in the usual print density for the image/text data, 
and at a more accurately-controlled, finer density for the 
separately-applied watermark data. (The latter may be 
effected as a slight modulation signal on the former.) This 
arrangement provides for essentially transparent integration 
into existing printer environments — no one need worry 
about the watermarking capability except the software apph- 
cations that specifically make use of same. 
Consumer Marking of Web-Based Materials 

Various items of printed media can originate off the web, 
yet be printed at home. Examples include movie tickets, 
coupons, car brochures, etc. Bedoop data can be added, or 
modified, by the software application or by the printer driver 
at the time of printing. (Altenoatively, the Bedoop data can 
be customized to correspond to the tiser before being down- 
loaded to the user's system for printing.) 

One advantage to Bedoop-encoding printed images 
locally, as opposed to Bedoop-encoding the image files prior 
to downloading for local printing, is that the encoding can be 
tailored in accordance with the particular properties of the 
local printer (e.g., to increase robustness or decrease 
visibility) — properties not generally known to a remote 
server. 

In one particular example, the UID field in the Bedoop 
data can be written with a value that serves as an index to a 
database of user profiles, permitting later systems to which 
the printed item is presented to personalize their response in 
accordance with the profile data. 

In another example, the UID field serves an authentication 
purpose, e.g., to verify that the printed medium actually was 
printed at a particular place, or by a particular user or at a 
particular time. 
Coffee Mug 

At retail coffee outlets, customers commonly order the 
same drink day after day ("half-decaf, short, skinny latte"). 
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Some customers present personal coffee mugs to the cashier, 
preferring the sensation of ceramic or metal to paper, and 
avoiding the trash/recycle dilemma. 
The drinker's "regular** order can be Bedoop-encoded 

5 either on the mug itself or, more commonly, on an adhesive 
label applied to the mug. The encoding can be in addition to 
other aesthetic imagery (e.g., artwork or a photo), or the 
marking can be purely data. Labels the size of postage 
stamps may be used. 

10 On handing the mug to the cashier, the customer can 
simply say "the regular." The cashier passes the mug in front 
of the optical scanning device of a Bedoop system associ- 
ated with the cash register. The system sleganographically 
decodes the data and provides the corresponding order 

15 ("half-decaf, short, skinny latte"), either textually or audibly 
(e.g., by a voice synthesizer) to the cashier or the barrista. 
The cash register system also knows the current price of the 
requested drink, and rings up the charge accordingly. 
Labels of the type described can be available to the 

20 cashier on pre-printed rolls, just as with other adhesive 
stickers, or can be printed on-demand, (Small label printers 
may be best suited in the latter case, given space constraints 
in retail outlets.) Customers ordering drinks for personal 
mugs may be invited to take a label corresponding to their 

25 just-ordered drink and apply it to their mug for future use. 
In variants on this basic theme, the mug label can be 
further encoded (or a supplemental label can be provided 
and encoded) with electronic payment information, such as 
the customer's credit card number, or the number of a debit 

30 account maintained by the coffee merchant for that cus- 
tomer. When the mug is scanned for the drink order, the 
system likewise detects the payment information and 
charges the corresponding fee to the appropriate account. 
(For security reasons, the system may be arranged so that the 

35 mug cannot be used to authorize more than, say $5 of coffee 
drink purchases per day.) 

In another variant on this theme, the system maintains an 
electronic log of coffee purchases made by the customer and, 
in accordance with then-prevailing marketing 

40 considerations, rewards the customer with a free drink after 
8 or 12, etc., drinks have been purchased. 

In still another variant on this theme, regular customers 
who use Bedoop-labeled mugs can participate in periodic 
promotions in which, for example, every N*'* such customer 

45 is rewarded with a cash or merchandise prize. Bells go off 
when the N'* mug is scanned. (N can be a fixed number, such 
as 500, or can be a random number — typically within a 
known range or with a known mean.) 
Smart Elevators 

50 In accordance with another embodiment of the invention, 
a building elevator is provided with one or more optical 
capture devices. Each device examines monitors the con- 
tents of the elevator chamber looking for Bedoop encoded 
objects, such as ID badges. 

55 On sensing a Bedoop-encoded object, the elevator can 
determine — among other data — the floor on which the wear- 
er's office is located. The system can then automaticaUy 
direct the elevator to that floor, without the need for the 
person to operate any buttons. (The elevator's button panel 

60 can be provided with a new, override button that can be 
operated to im-selecl the most recently selected floor(s), e.g., 
in case a user wants to travel to a different floor.) To aid in 
identification, the Bedoop objects (e.g., badges) can be 
colored a distinctive color, permitting the system to more 

65 easily identify candidate objects fi-om other items within the 
optical capture devices' field of view. Or the object can be 
provided with a retro-reflective coating, and the elevator can 
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be equipped with one or more illuminalion sources of known 
spectral or temporal quality (e.g., coostant infra red, or 
constant illumination with a single- or multi-line spectrum, 
or a pulsed light source of known periodicity, LEDs or 
semiconductor lasers, each with an associated dififuser, can 5 
be used for each the foregoing and can be paired with the 
image capture devices). Other such tell-tale clues can like- 
wise be used to aid in object location. In all such cases, the 
optical capture device can sense the tell-tale clue(s) using a 
wide field of view sensor. The device can then be physically lO 
or electronically steered, and/or zoomed, to acquire a higher 
resolution image of the digitally-encoded object suitable for 
decoding. 
Magazines 

Magazine (and newspaper) pages can be steganographi- 15 
cally encoded with Bedoop data to provide another "paper as 
portal" experience. As with the earlier described office 
document case, the encoded data yields an address to a 
computer location (e.g., a web page) having the same, or 
related, content, 20 

In one exemplary embodiment, the blank magazine page 
stock is Bedoop-cncoded prior to printing. The watermark- 
ing can be performed by high speed ink-jet devices, which 
splatter a fine pattem of essentially imperceptible ink drop- 
lets across each page. Each page can be differently water- 25 
marked so that, on decoding, page 21 of a magazine can be 
distinguished from page 22 of the same magazine (and page 
106 of the Jun. 21, 1999, issue can be distinguished from 
page 106 of the Jun. 28, 1999, issue). If desired, each page 
can be further segregated into regions — either in accordance 30 
with the actual boundaries of articles that will later be 
printed on the pages, or in a grid pattern, e.g., of 3 columns 
across by 5 rows high. Each region conveys a distinct 
Bedoop code, permitting different portions of the page to 
lead to different web data.) 35 

After watermarking and printing, the pages thus produced 
are bound in the usual fashion with others to form the 
finished magazine. (Not all pages in the magazine need to be 
watermarked.) 

Of course, the watermarking can be effected by processes 40 
other than ink-jet printing. For example, texturing by pres- 
sure rollers is another option well suited for the large 
volumes of paper to be processed. 

On presenting a magazine to the optical scanner device of 
a Bedoop-compliant computer, the computer senses the 45 
Bedoop data, decodes same, and launches a web browser to 
an internet address corresponding to the Bedoop data. If the 
magazine page is an advertisement, the internet address can 
provide information complementary to the advertisement. 
For example, if the magazine page is an advertisement for a 50 
grocery item, the Bedoop data can identify a web page on 
which recipes using the advertised item are presented. If the 
magazine page includes a photo of a tropical beach, the 
Bedoop data can lead to a travel web page (e.g., hosted by 
Expedia or other travel service) that presents fare and 55 
lodging information useful to a reader who wants to vacation 
at the illustrated beach. (The fare information can be cus- 
tomized to the reader's home airport by reference to user 
profile data stored on the iiser's computer and relayed to the 
web site to permit customization of the displayed page.) 60 

The data to which the Bedoop data leads needn't be static; 
it can be updated on a weekly, daily, or other basis. Thus, if 
a months-old magazine page is presented to a Bedoop 
device, the resultant data can be up-to-the-minute. 

In the case of advertising, the inclusion of Bedoop data 65 
increases the value of the ad to the advertiser, and so merits 
a higher charge to the advertiser from the magazine pub- 
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Usher. This higher charge may be shared with the enterprise 
(s) that provides the Bedoop technology and infrastructure 
through which the higher value is achieved. 
Business Card Applications 

Conventional business cards can be steganographicaUy 
encoded with Bedoop data, e.g., by texturing, watermark 
tinting, ink-jet splattering, text steganography, etc. As with 
many of the earlier-described embodiments, the stegano- 
graphic encoding is tailored to facilitate decoding in the 
presence of arbitrary rotation or scale distortion of the card 
introduced during scanning. (Some such techniques are 
shown, e.g., in applicant's related patents identified above. 
Various other techniques are known to artisans.) 

When a recipient of a business card holds it in front of a 
Bedoop sensor, the operating system on the local system 
launches a local Bedoop application. That local Bedoop 
application, in turn, establishes an external internet connec- 
tion to a remote business card server. The address of that 
server may already be known to the local Bedoop applica- 
tion (e.g., having been stored fi-om previous use), or the local 
Bedoop system can traverse the above-described public 
network of DNS servers to reach the business card server. 

A database on the business card name server maintains a 
large collection of business card data, one database record 
per UID. When that server receives Bedoop data from a local 
Bedoop system, it parses out the UID and accesses the 
corresponding database record. This record typically 
includes more information than is commonly printed on 
conventional business cards. Sample fields from the record 
may include, for example, name, title, ofl&ce phone, office 
fax, home phone, home fax, cellular phone, email address, 
company name, corporate web page address, personal web 
page address, secretary's name, spouse's name, and birth- 
day. This record is transmitted back to the originating 
Bedoop system. 

The local Bedoop system now has the data, but needs 
further instruction from the user as to how it should be 
processed. Should a telephone ntmiber be dialed? Should the 
information be entered into a personal contact manager 
database (e.g.. Outlook) on the local system? Etc. 

In an exemplary embodiment, the local system presents 
the available choices to the user, e.g., by textual prompts, 
synthesized voice, etc. The user responds by manipulating 
the business card in a manner prompted by the system (e.g., 
move down to telephone at office; move up to telephone at 
home; move right to access corporate web page; move left 
to access personal web page; rotate left to enter certain 
elements from the database record (filtered in accordance 
with a template) into personal contact manager database, etc. 
The local Bedoop system responds accordingly. 

Some card givers may choose to make additional infor- 
mation available to card recipients — ^information beyond 
that known in prior art contact-management software appli- 
cations. For example, one of the choices presented by a local 
Bedoop system in response to presentation of a business 
card may be to review the card -giver's personal calendar. 
(The card-giver can maintain his or her personal calendar on 
a web-accessible computer.) By such arrangement, the card- 
recipient can learn when the card-giver may be foimd in the 
office, when appointments might be scheduled, etc., etc. 

Typically, access to this web-calendar is not available to 
casual web browsers, but is accessible only in response to 
Bedoop data (which may thus be regarded as a form of 
authentication or password data). 

Some users may carry several differently-encoded cards, 
each with a different level of access authorization (e.g., with 
different UIDs). Thus, some cards may access a biographical 
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page without any calendar information, other cards may 
access the same or different page with access enabled to 
today's calendar, or this week^s calendar, only, and still other 
cards (e.g., the "spouse" card) may access the same or 
different page with access enabled for the card-giver's 5 
complete calendar. The user can distribute these different 
cards to different persons in accordance with the amount of 
personal information desired to be shared with each. 

In accordance with a related embodiment, the database 
record corresponding to Bedoop business card data can lO 
include a "now" telephone number field. This field can be 
continually-updated throughout the day with the then-most- 
suitable communications channel to the card-giver. When 
the card-giver leaves home to go to the office, or leaves the 
office for a trip in the car, or works a week at a corporate 15 
office in another town, etc., this data field can be updated 
accordingly. (A pocket GPS receiver, with a wireless uplink, 
can be carried by the person to aid in switching the "now" 
number among various known possibilities depending on 
the person's instantaneous position.) When this database 20 
record is polled for the "now" number, it provides the 
then-current information. 

Consider a Bedoop-enabled public telephone. To dial the 
phone, a business card is held in front of the Bedoop sensor 
(or slid through an optical scanner track). Hie phone inter- 25 
rogates the database at the business card server for the 
"now" number and dials that number. 

To update the any of the fields stored in the database 
record, the card giver can use a special card that provides 
write-authorization privileges. This special card can be a 30 
specially encoded version of the business card, or can be 
another object unique to the card-giver (e.g., the card-giver's 
driver's license). 

The reference to business cards and personal calendars is 
illustrative only. Going back a century, "calling cards" were 35 
used by persons whose interests were strictly social, rather 
than business. The just-discussed principles can be similarly 
applied. Teenagers can carry small cards to exchange with 
new acquaintances to grant access to private dossiers of 
personal information, favorite music, artwork, video clips, 40 
etc. The cards can be decorated with art or other indicia that 
can serve purposes wholly unrelated to the Bedoop data 
steganographically encoded therein. 
Gestural Input 

A Bedoop system can determine the scale state, rotation 45 
state, X-Y offset, and differential scale state, of an object by 
reference to embedded calibration data, or other techniques. 
If the scan device operates at a suitably high frame rate (e.g., 
five or ten frames per second), change(s) in any or all of 
these four variables can be tracked over time, and can serve 50 
as additional input. 

In an earlier-discussed example, moving an object to the 
left or right in front of the Bedoop scanner caused a left- or 
right-positioned button in a dialog box to be selected. This 
is a change in the X-Y oflket of the scanned object. In that 55 
earlier example, moving the object inwardly towards the 
camera caused the selected button to be activated. This is a 
change in the scale state of the scanned object. 

In similar fashion, twisting the object to the left or right 
can prompt one of two further responses in a suitably 60 
programmed Bedoop application. (This is a change in the 
rotation state.) Likewise, tilting the object so that one part is 
moved towards or away from the camera can prompt one of 
two further responses in the application. (This is a change in 
the differential scale state.) 65 

In the business card case just-discussed, for example, the 
card can be held in front of the Bedoop scanner of a 
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computer. If the card is twisted to the left, the computer 
opens a web browser to a web page address corresponding 
to Bedoop data on the card. If the card is twisted to the right, 
the computer opens an e-mail template, pre-addressed to an 
e-mail address indicated by the card. 

In other examples, twisting an object to move the right 
edge towards the scanner can be used to effect a right mouse 
click input, and twisting the object to move the right edge 
away from the scanner can be used to effect a left mouse 
click input. 

Simultaneous changes in two of these four positioning 
variables can be used to provide one of four different inputs 
to the computer (e.g., (a) twisting left while moving in; (b) 
twisting left while moving out; (c) twisting right while 
moving in; and (d) twisting right while moving out). Simul- 
taneous changes to three or all four of these variables can 
similarly be used to provide one of eight or sixteen different 
inputs to the computer. 

Simultaneous manipulations of the object in two or more 
of these modes is generally imwieldy, and loses the simple, 
intuitive, feel that characterizes manipulation of the object in 
one mode. However, a similar effect can be achieved by 
sequential, rather than simultaneous, maniptdation of the 
card in different modes (e.g., twist left, then move in). 
Moreover, sequential manipulations permit the same mode 
to be used twice in succession (e.g., move in, then move 
out). By such sequential manipulations of the object, arbi- 
trarily complex input can be conveyed to the Bedoop 
system. 

(It will be recognized that a digitally-encoded object is not 
necessary to the gestural-input applications described above. 
Any object (talisman) that can be distinguished in the image 
data can be manipulated by a user in the manners described 
above, and an appropriate system can recognize the move- 
ment of the object and respond accordingly. The provision of 
digital data on the object provides a further dimension of 
functionality (e.g., permitting the same gesture to mean 
different things, depending on the digital encoding of the 
object being manipulated), but this is not essential. 

Moreover, even within the realm of digitally-encoded 
gestural talismans, steganographic encoding is not essential. 
Any other known form of optically-recognizable digital 
encoding (e.g., ID and 2D bar codes, etc.) can readily be 
employed. 

In an illustrative embodiment, a business card or photo- 
graph is used as the talisman, but the range of possible 
talismans is essentially uinlimited. 
Gestural Decoding Module 

There are various ways in which the Bedoop system's 
decoding of gestural input can be effected. In some Bedoop 
systems, this functionality is provided as part of the Bedoop 
applications. Generally, however, the applications must be 
provided with the raw frame data in order to discern the 
gestural movements. Since this functionahty is typically 
utilized by many Bedoop applications, it is generally pref- 
erable to provide a single set of gestural interpretation 
software fiinctions (commonly at the operating system level) 
to analyze the frame data, and make available gestural 
output data in standardized form to all Bedoop applications. 

In one such system, a gestural decoding module tracks the 
encoded object within the series of image data frames, and 
outputs various parameters characterizing the object's posi- 
tion and manipulation over time. IWo of these parameters 
indicate the X-Y position of the object within current frame 
of image data. The module can identify a reference point (or 
several) on the object, and output two corresponding posi- 
tion data (X and Y). The first represents the horizontal offset 
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of the reference point from the center of the image frame, horizontal). (The two reference points must be dislinguish- 

represented as a percentage of frame width. A two's comple- able from each other regardless of angular position of the 

ment representation, or other representation capable of object, if data in the full range of 0-360 degrees is to be 

expressing both positive and negative values, can be used so represented. If these two points are not distinguishable, it 

that this parameter has a positive value if the reference point 5 may only be possible to represent data in the range of 0-180 

is right of center-frame, and has a negative value if the degrees.) As before, a dynamic parameter C can also be 

reference point is left of center frame. The second parameter, discerned to express the change in the rotation state param- 

Y, similarly characterizes the position of the reference point eter C since the last frame. This parameter can be in seven 

above or below center-frame (with above-being represented bit, two's complement form, with positive values indicating 

by a positive value). Each of these two parameters can be lO change in a clockwise rotation 

expressed as a seven-bit byte. A new pair of X,Y parameters The foregoing analysis techniques, and representation 

is output from the gestural decoding module each time a new metrics, are of course illustrative only. The artisan will 

frame of image data is processed. recognize many other arrangements that can meet the needs 

In many applications, the absolute X-Y position of the of the particular Bedoop applications being served, 
object is not important. Rather, it is the movement of the is In the illustrative system, the Bedoop application pro- 
object in X and Y from frame-to-frame that controls some grams communicate with the gestural decoding module 
aspect ofthe system's response. The Bedoop application can through a standardized set of interface protocols, such as 
monitor the change in the two above-described parameters, APIs. One API can query the gestural input module for some 
frame to frame, to discern such movement. More commonly, or all of the current position parameters (e.g., any or all of 
however, the gestural decoding module performs this func- 20 X, Y, A, B, and C). The module responds to the calhng 
tion and outputs two further parameters, X' and Y". The application with the requested parameter(s). Another API 
former indicates the movement of the reference point in can query the gestural input module for some or all of the 
right/left directions since the last image frame, as a percent- ciirrent movement data (e.g., any or all of X', Y, A', B' and 
age of the full-frame width. Again, this parameter is repre- C). Still another API can request the gestural decoding 
sented in two's complement form, with positive values 25 module to provide updated values for some or all of the 
representing movement in the rightward direction, and nega- position or movement data on a running basis, as soon as 
tive values representing movement in the leftward direction. they are discerned from each frame. A complementary API 
The later parameter similarly indicates the movement of the discontinues the foregoing operation. By such arrangement, 
reference point in up/down directions since the last frame. all of the gestural data is available, but the Bedoop appli- 

The scale, differential scale, and rotation states of the 30 cation programs only obtain the particular data they need, 

object can be similarly analyzed and represented by param- and only when they ask for it. 

eters output from the gestural decoding module. In Bedoop applications that communicate with external 

Scale state can be discerned by reference to two (or more) servers, just the Bedoop data (i.e., CLASS, DNS, and 

reference points on the object (e.g., diagonal comers of a optionally UID) may initially be sent. If the remote server 

card). The distance between the two points (or the area 35 needs to consider gestural data in deciding how to respond, 

circumscribed by three or more points) is discerned, and the remote server can poll the local Bedoop system for the 

expressed as a percentage of the diagonal size of the image necessary data. The requested gestural data is then sent by 

frame (or its area). A single output parameter, A, which may the local Bedoop system to the remote server in one or more 

be a seven-bit binary representation, is output. separate transmissions. 

As with X-Y data, the gestural decoding module can 40 In other embodiments, since the gestural data is of such 

likewise monitor changes in the scale state parameter since low bandwidth (e.g., roughly 56 bits per image frame), it 

the last frame, and product a corresponding output parameter may routinely and automatically be sent to the remote 

A'. This parameter can be expressed in two's complement computer, so that the gesture data is immediately available 

form, with positive values indicating movement of the in case it is needed. In an illustrative implementation, this 

object towards the sensor since the last frame, and negative 45 data is assembled into an 8-byte packet, with the first byte 

values indicating movement away. of the packet (e.g., the X parameter) being prefixed with a 

A differential scale parameter, B, can be discerned by "1" sync bit, and subsequent bytes of the packet being 

reference to four reference points on the object (e.g., center prefixed with "0" sync bits. (The sync bits can be used to aid 

points on the four edges of a card). The two points on the in accurate packet decoding.) 

side edges of the card define a horizontal line; the two points so In some embodiments, it is useful to provide for an 

on the top and bottom edges of the card define a vertical line, extension to the normal 64-bit Bedoop lengtfi to accommo- 

The ratio of the two line lengths is a measure of differential date an associated packet of gestural data. This can be 

scale. This ratio can be expressed as the shorter line's length effected by use of a reserved bit, e.g., in the UID field of the 

as a percentage of the longer line's length (i.e., the ratio is Bedoop packet. This bit normally has a "0" value. If it has 

always between zero and one). Again, a two's complement 55 a "1" value, that indicates that the Bedoop data isn't just the 

seven-bit representation can be used, with positive values usual 64 bits, but instead is 128 bits, with the latter 64 bits 

indicating that the vertical line is shorter, and negative comprising a packet of gestural data, 

values indicating that the horizontal line is shorter. (As Similar extension protocols can be used to associate other 

before, a dynamic parameter B' can also be discerned to ancillary data with Bedoop data. A different reserved bit in 

express the change in the differential scale parameter B since 60 the UID field, for example, may signal that a further data 

the last frame, again in two's complement, seven bit form.) field of 256 bits follows the Bedoop data — a data field that 

A rotation state parameter C can be discerned by the will be interpreted by the remote computer that ultimately 

angular orientation of a line defined by two reference points services the Bedoop data in a known manner. (Such bits may 

on the object (e.g., center points on the two side edges of a convey, e.g., profile data, credit card data, etc.) The 

card). This parameter can be encoded as a seven -bit binary 65 appended data field, in turn, may include one or more bits 

value representing the percentage of rotational offiset in a signaling the presence of still further appended data 

clockwise direction from a reference orientation (e.g., Grandmothers 
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It is a common complainl thai computers are too complex address at which video data from a web cam at the pictured 

for most people. Attempts to simplify computer-user inter- person's home is presented. Rotating the photo clockwise 

action to facilitate use by less experienced users usually may present an e-mail form, pre-addressed to the e-mail 

serve to frustrate more experienced users. address of the depicted person. Moving the photo to the right 

In accordance with another embodiment of the present 5 may query a database on the system for other photographs 

invention, the sophistication of a computer user is stegano- depicting the same individual or subject, which can be 

graphically indicated on a talisman used by that user to presented in response to further user input. Etc. 

interact with the system. The computer detects this In this and other embodiments, it is helpful for the Bedoop 

steganographically-encoded data, and alters its mode of device to prompt the user to aid in manipulating the object, 

interacting with the user accordingly. lO This can be done audibly (e.g., "move photo left to dial at 

Consider internet browser software. Experienced users home") or by visual clues (e.g., presenting left- or right- 
are familiar with the different functionality that can be pointing arrows). 

accessed, e.g., by various drop-down menus/sub-menus, by Bedoop data in photographs can also be used to annotate 

the keyboard shortcuts, by the menus available via right- the photographs, as with notes on the back of a photograph, 

clicking on the mouse, by manipulating the roller mouse 15 or printed under the photograph in a photo album. The 

scroll wheel and scroll button, etc., etc. Grandmothers of Bedoop data can lead to a remote database, where the 

such users, typically, are not so familiar. photograph owner is permitted to enter a textual (or audio) 

Although gestural interfaces hold great promise for sim- narrative in association with each photograph's UID. Years 

plifying user-computer interaction, the same dichotomy later, when some of the names have been forgotten, the 

between experienced users and inexperienced users is likely 20 photograph can be positioned in front of a Bedoop sensor, 

to persist, frustrating one class of user or the other. and the system responds by providing the annotation pro- 

To help close this gap, a computer system according to vided by the photograph owner years earlier, 

this embodiment of the invention responds to gestures in Drivers Licenses and Other Cards 

different manners, depending on the expertise level indi- Drivers licenses, social security cards, or other identity 

cated by encoding of the talisman. For an expert user, for 25 documents may be encoded by the issuing authority with 

example, the gestural interface active in the internet browser Bedoop data that permits access to the holder's personal 

software may display the stored list of Favorite web records over the web. On presenting the document to a 

addresses in response to tipping the left edge of the talisman Bedoop system, the system directs a web browser to a 

towards the optical sensor. Once this list is displayed, the private address corresponding to data encoded on the docu- 

expert user may rotate the talisman to the right to cause the 30 ment. At that address, the holder of the document can review 

highlighting to scroll down the list from the top. Rotating the governmental records, such as state or federal tax redira 

talisman to the left may scroll the list of Favorites up from data, social security entitlements, etc., as well as privately- 

the bottom. The speed of scrolling can be varied in accor- maintained records, such as credit records, etc. User selec- 

dance with the degree of rotation of the talisman from a tion among various functions can be effected by spatial 

default orientation. 35 manipulation of the document. (Entry of additional data. 

In contrast, for the novice user, these talisman manipula- such as social security number or mother's maiden name, 

tions may be confounding rather than empowering. Tipping may be required of the user to assm-e privacy in case the 

the left edge of the talisman towards the sensor may occur document is lost or stolen.) 

as often by mistake as on purpose. For such users, a more By manipulating a driver's license in front of a Bedoop 

satisfactory interface may be provided by relying on simple 40 sensor, a user can request renewal of the driver's license, and 

X-Y movement of the talisman to move an on-screen cursor, authorize payment of the corresponding fee. 

with a movement of the taUsman towards the sensor to serve Bank cards (debit, credit, etc.) can similarly be encoded 

as a selection signal (i.e., like a left-mouse click). with Bedoop data to permit the holder to access bank records 

(In the examplejust-cited, the expert user summoned a list corresponding to the bank card account. (Entry of a PIN 

of Favorite web sites. Different "Favorites" lists can be 45 code may be required to assure privacy.) 

maintained by the computer — each in association with dif- Such documents can also be used to access other personal 

ferent talismans. A husband who uses one talisman is data. One example is e-mail. A traveler might pause at a 

provided a different "Favorites" list than a wife who uses a Bedoop kiosk at an airport and present a driver's license, 

different talisman.) Without anything more, the kiosk may present email that is 

Printed Kctures 50 waiting for the traveler on an associated display screen. 

In accordance with this aspect of the invention, a printed On recognizing a driver's license, the kiosk can access a 
photograph can be steganographically encoded with Bedoop remote site (which may be maintained by the Department of 
data leading to information relating to the depicted person Motor vehicles, another government entity, a private entity, 
(e.g., contact information, biographical information, etc.). or by the traveler), authenticating the operation by present- 
Such a photograph can be presented to a Bedoop sensor 55 ing Bedoop data encoded on the Ucense, and obtaining 
on a telephone. In a simple embodiment, the telephone information that the person has pre-approved for release in 
simply processes the Bedoop data to obtain a corresponding response to such authorized access. This information can 
default telephone number, and dials the number. In other include e-mail account and password information. Using 
embodiments, various options are possible, e.g., dial home this information, the kiosk queries the corresponding e-mail 
number or dial work number. On presenting the photograph 60 server, and downloads a copy of recently received mail for 
to the telephone, for example, moving the photo to the left presentation at the kiosk. (A user-entered PIN number may 
may dial the person at home, while moving he photo to the be required at some point in the process, e.g., in querying the 
right may dial the person at work. remote site for sensitive e-mail password data, before pre- 
As telephones evolve into more capable, multi-function senting the downloaded e-mail for viewing, etc., to ensure 
devices, other manipulations can invoke other actions, in a 65 privacy.) 

computer/telephone hybrid device, for example, rotating the Other cards carried in wallets and purses can also be 

photo counterclockwise may launch a web browser to an encoded to enable various functions. The local sandwich 
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shop that rewards regular customers by awarding a free towards camera to order the product; move away from 

sandwich after a dozen have been purchased can encode camera for product information. If the object is moved 

their frequent-buyer card with Bedoop data leading to the towards the camera to effect an order, the user can be 

shop's web-based sandwich delivery service. Or the prompted to further manipulate the object to specify delivery 

frequent-buyer card can be eliminated, and customers can 5 options (e.g., rotate left for overnight mail, rotate right for 

instead wave their business card or other identity document regular mail). If the object is moved away from the camera 

in front of the shop's Bedoop sensor to get purchase credit to request product information, the user can be promoted to 

in a tally maintained by the sandwich shop's computer. further manipulate the object to specify the type of infor- 

Food stamps, health insurance cards, and written medical mation desired (e.g., rotate left for recipes, rotate right for 

prescriptions, can likewise be encoded with digital data to lO FDA nutritional information, move up for information on 

enable the provision of new fiinctionality, other products in this family, move down to send an email 

At large trade shows, such as COMDEX, vendors needn't to the product manufacturer), 

publish thick, glossy brochures to hand out to visitors. Credit card or other customer billing information. 

Instead, they may print various stylish promo cards for together with mailing address information, can be stored in 

distribution. When later presented to a Bedoop sensor, each 15 a profile on the Bedoop system, and relayed to the transac- 

card leads to a web-based presentation — optionally includ- tional web site either automatically when a purchase action 

ing persuasive video and other multi-media components. is invoked, or after the user afBrms that such information 

The user can be prompted to provide data to customize, or should be sent (which affirmation may be signaled by 

focus, the presentation to the user's particular requirements. manipulation of the packaging or advertisement in one of the 

If the user wants further information, a request can be made 20 earlier-described modes). Other modes of payment can 

by the click of a mouse (or the twist of a card). naturally be employed. (One such alternative is the first-to- 

Prizes and Product Promotions redeem electronic money system described in the present 

Product packaging (e.g., Coke cans, Snapple bottles, assignee's patent application 60/134,782.) 

Pepsi 12-pack boxes) can be encoded for contest purposes. Clothing 

The encoding can be customized, item to item, so that 25 In accordance with another aspect of the invention, cloth- 
selected items — ^when Bedoop scanned — are recognized to ing can be ordered on-line by presenting to a Bedoop system 
be the one in a hundred that entitles the owner to a cash or a photograph from a catalog, or a garment tag or label, 
merchandise prize, A remote server to which the item's Encoded on each is product-identifying data, including a 
Bedoop data is provided queries the user for contact infor- manufacturer ID. The Bedoop system responds by estab- 
mation (e.g., address, phone number) so the prize can be 30 lishing a link to a remote computer maintained by or on 
awarded or, for smaller prizes, the system can print out an behaff of the manufacturer. In addition to relaying the 
award certificate redeemable at local merchants for products product identification data to the remote computer, the 
or cash. Once a winning item is identified to the remote Bedoop application also sends some or all of a clothing 
server, its UID on the server is marked as redeemed so that profile maintained by the user on the local computer. This 
the item cannot later be presented to win another prize. 35 profile can specify, e.g., the person's weight, height, shoe 
In other such embodiments, all of the items are encoded size, waist size, inseam, etc. The remote computer can 
identically. Winners are determined randomly. For example, confirm availability of the identified item in the size speci- 
during a contest period, persons around the world may fied in the clothing profile, and solicit payment and shipping 
present Coke cans to Bedoop systems. The corresponding instructions. 
Bedoop application on each user computer submits Bedoop 40 Computer Access Cards 

data to a corresponding web address. The user's e-mail This disclosure earlier considered access cards used to 

address may also be included with the submission. As this gain access to secure buildings. Related principles can be 

data is relayed to the corresponding server computer(s), used in conjunction with computer access, 

every N''' set of data is deemed to be a wirmer, and a A driver's license, employee photo ID, or other such 

corresponding award notification or prize is dispatched to 45 document can be presented to a Bedoop sensor on a com- 

the Bedoop system from which the winning set of data puter. The computer recognizes the user and can take various 

originated. steps in response. 

The server computer that receives such contest submittals One response is to log onto a network. Another is to set 

from client Bedoop systems can be arranged to prevent a load a user profile file by which the computer knows how to 

single user from bombarding the server with multiple sets of 50 arrange the desktop in the user's preferred manner. By 

data in an attempt to win by brute force. (This may be done, manipulating the Bedoop-e needed object, the user can fur- 

for example, by checking the included e-mail address, and ther vary the environment (e.g., rotate left to launch standard 

not considering a data submittal if the same e-mail address business productivity applications and software develop- 

was encountered in data submitted within the past hour. ment applications; rotate left to launch lunchtime 

Similar anti-brute-force protection can be provided on the 55 diversions — stock update, recreational games, etc.). 

user'scomputer,preventing,e.g., repeated contest data to be Hotel rooms are increasingly providing computer ser- 

sent more frequently than once per hour. More sophisticated vices. By presenting a driver's license, a Bedoop-equipped 

anti-brute-force measures can of course be provided.) computer in a hotel room can link to a remote site indicated 

Product Information and Ordering by the Bedoop data, obtain preference data for that user, and 

In accordance with another embodiment of the present 60 launch applications on the hotel computer in an arrangement 

invention, product packaging and product advertisements that mimics that user's familiar work computer environ- 

can be encoded with Bedoop data that, when presented to a ment. 

Bedoop system, initiates a link to a web page from which AudioA^ideo Disks, Software, and Books 

that product can be purchased, or more information Bedoop data can be conveyed by indicia or texturing on 

obtained. Once the link has been established, the user can be 65 the surfaces of CD and DVD disks, on the labels (or 

instructed to manipulate the object in different of the earlier- authenticity certificates) for same, on the enclosures for 

described modes to effect different functions, e.g., move same (e.g., jewel box, plastic case, etc.), on book dust 
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jackets, on book pages, etc. Any of these objects can be 
presented to a Bedoop device to establish a hnk to a related 
web site. The consumer can then manipulate the object (or 
otherwise choose) to select different options. 

For music, one option is to receive MP3 or other clips of 5 
songs by the same artist on other CDs, or of songs from other 
artists of the same genre. Another is to view music video 
clips featuring the same artist. Still another is to order tickets 
to upcoming concerts by that artist. In-store kiosks can 
permit tentative customers to listen to sample tracks before lO 
they buy. 

Similar options can be presented for video DVDs. In the 
case of video, this can include listings of other movies with 
the same director, with the same star(s), etc. In the case of 
software, the options can include advisories, bug fixes, 15 
product updates and upgrades, etc. Naturally, the user can 
make purchases from these sites, e.g., of other music by the 
same artist, other videos with the same star, software 
upgrades, etc. 

Similar options can be accessed using Bedoop data asso- 20 
ciated with printed book materials. 
Ad Tracking 

Advertisers commonly use different advertisements for 
the same product or service, and employ means to track 
which ad is more effective within which demographic group. 25 
Bedoop can provide such functionality. 

Consider a travel service web site that is promoting 
Hawaiian vacations. Bedoop data from several advertise- 
ments can lead consumers to the site. 

Identical advertisements can be placed in several different 30 
magazines. Each is encoded with a different Bedoop UID. 
By monitoring the UIDs of the Bedoop inquiries to the site, 
the travel service can determine which magazines yield the 
highest consumer response (e.g., per thousand readers). 

Likewise, within a single magazine, two or more adver- 35 
tisements may be encoded with Bedoop data leading to the 
site — again, each with a different UID. Again, analysis of the 
UIDs used in accessing the site can indicate which adver- 
tisement was the more effective. 

The instantaneous nature of the internet links permits 40 
advertisers to learn how consumer responses to print adver- 
tisements vary with time-of-day, yielding information that 
may assist in making ads for certain products more effective. 

More elaborate variants and combinations of the forego- 
ing are, of course, possible, If the consumers provide per- 45 
sonal information in response to the ads (either by permit- 
ting access to pre-stored personal profile data, or by filling 
in web-based forms, or by manipulation of the ad (e.g., 
"please move the ad towards your Bedoop sensor if you 
drank coffee this morning*')), still richer statistical data can 50 
be gleaned. 
Rolodex of Cards 

Bedoop -encoded business cards as detailed above can be 
accumulated and kept near a telephone or computer in a 
Rolodex-like arrangement. If a refrigerator ice-maker 55 
malfunctions, a homeowner can find the card for the appli- 
ance repairman used a few years ago, and present it to a 
Bedoop sensor. A link is established to the repairman's 
company (e.g., web site or via telephone). At a web site, the 
repairman may provide basic information, such as hours of 60 
availability, current fee schedule, etc. The homeowner may 
select an option (by card gesture or otherwise) to invoke a 
teleconference (e.g., NetMeeting) to consult about the prob- 
lem. Or the homeowner may select another option to send 
e-mail. Still a further option may permit the homeowner to 65 
schedule a house call on the repairman's weekly calendar. 
Still a further option may permit the homeowner to view one 
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or more short videos instructing customers how to fix certain 
common appliance problems. 
Stored Value Cards 

The earlier cited "first- to-redeem" electronic money sys- 
tem may encode Bedoop data on a card that leads to storage 
at which the random-number tokens (which represent incre- 
ments of money) are stored. Presenting the card to a Bedoop 
system latmcbes an application that reads and encrypts the 
tokens and forwards the encrypted data to the clearinghouse 
computer of the corresponding bank to learn their remaining 
value. There the tokens are decrypted and checked for 
validity (but not redeemed). The bank computer responds to 
the Bedoop system, indicating the remaining value of the 
tokens on the card. 

For security reasons, the storage containing the random- 
number tokens should not be generally accessible. Instead, 
the tiser must provide authentication data indicating autho- 
rization to gain access to that information. This authentica- 
tion data may be a PIN code. Or the user may provide 
authentication by presenting a second Bedoop-encoded 
object, e.g., a driver's license to the Bedoop system. (Many 
other Bedoop systems may advantageously tise, or require 
the use of, two or more Bedoop objects — either presented 
one after the other, or all at the same time. The Bedoop 
system can provide visual or audible prompts leading the 
user to present the further Bedoop object(s) as necessary. 
Ski Lift Tickets 

In accordance with another embodiment, ski lift tickets 
are Bedoop encoded to provide, various functionaUty. 

For example, instead of buying a lift ticket good for a day, 
a skier may purchase a ticket good for eight lifts. This data 
is encoded on the ticket, and sensed by a Bedoop sensor at 
each lift. The sensors are networked to a common server that 
tracks the number of lifts actually purchased, and updates 
the number as used. The skier is informed of the number of 
rides remaining on entering or leaving the lift. Statistical 
data can be collected about trail usage (e.g., N % percent of 
skiers ski all day along just two lifts, etc.). 

Off the slopes, back at home, the used lift ticket may be 
presented to a Bedoop sensor to obtain current snow con- 
ditions and Ifft hours, or to review trail maps, or to order ski 
vacation packages. If the ticket is encoded with the owner's 
name, UID, or other information of commercial/marketing 
interest, local merchants may give the bearer discounts on 
selected goods in response to Bedoop scanning of the ticket 
and recovery of such information. 
REI Membership Cards 

Membership cards for certain stores can be Bedoop- 
encoded to provide added value to the member. For outdoor 
gear stores such as REI, presentation of the card to a Bedoop 
sensor can lead to a library of USGS maps, to web pages 
with current fishing and hunting regidations, etc. Naturally, 
the store's on-line ordering site is just a quick twist away. 
Theme Park Tickets 

Theme park tickets can be encoded with the age and 
gender of the visitor, and with additional data permitting the 
experience to be customized (e.g., from a roster of theme 
park personalities, the visitor's favorite is Indiana Jones). 
Throughout the park are kiosks to which the visitor can 
present the ticket to orchestrate the visit to follow a particu- 
lar story line. Some kiosks issue premiums matching the 
age/gender of the recipient. 
Car Keys 

In accordance with another embodiment of the invention, 
car keys (or key ring fobs) are Bedoop encoded. When the 
car is taken to a shop for service, the mechanic presents the 
key to a Bedoop sensor, and thereby obtains the car's 
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maintenance history from a remote server on which it is 
maintained. At home, the key can be presented to a Bedoop 
sensor and manipulated to navigate through a variety of 
automotive-related web sites. 

In some embodiments, the Bedoop-encoded object is not 5 
used to navigate to a site, but is instead used to provide data 
once a user's computer is otherwise linked to a web site, A 
user surfing the web who ends up at a car valuation site can 
present a key to the Bedoop scanner. The Bedoop data is 
used to access a remote database where the make, model, 30 
options, etc., of the car are stored. This data is provided to 
a database engine that returns to the user the estimated value 
of the car. 

While visiting a mechanic's web site, presentation (and 
optionally manipulation) of a key or key ring fob can be 15 
employed to schedule a service appointment for the car. 
Fashion Coordination 

Some department stores and clothing retailers offer "per- 
sonal shoppers" to perform various services. For example, a 
customer who is purchasing a dress may ask a personal 20 
shopper for assistance in selecting shoes or accessories that 
complement the dress. 

A Bedoop-encoded garment tag on the dress can be 
employed to obtain similar assistance. In response to such a 
tag, a Bedoop system can query a database to obtain a 25 
mini-catalog of clothes and accessories that have previously 
been identified as complementing the dress identified by the 
tag. These items can be individually displayed on a screen 
associated with the system, or a virtual model wearing the 
dress — together with one or more of the recommended 30 
accessories — can be synthesized and depicted. The shopper 
may quickly review the look achieved by the model wearing 
the dress with various different pairs of shoes, etc., by 
repeatedly activating a user interface control (by mouse, 
touch screen, or garment tag gestures) to cycle through 35 
different combinations. 

A shopper's credit card can be Bedoop-encoded so as to 
lead Bedoop systems of particular stores (i.e., stores pre- 
authorized by the shopper) to a profile on the shopper (e.g., 
containing size information, repeat purchase information, 40 
return history, style/color preferences, etc). 
Credit Card Purchases 

When a consumer visits a commercial web site and 
wishes to purchase a displayed product, the transaction can 
be speeded simply by presenting a Bedoop-encoded credit 45 
card to a Bedoop sensor on the user's computer. The Bedoop 
data on the card leads to a database entry containing the 
credit card number and expiration date. The Bedoop appli- 
cation then sends this information (optionally after encrypt- 
ing same) to the web site with instructions to purchase the 50 
depicted product. 

(Impulse purchases are commonly deterred by the hurdles 
posed between the purchase impulse and the completed 
purchase. This and other Bedoop applications aid in reduc- 
ing such hurdles.) 55 
Product Marketing 

Bedoop data relating to one product or service can be used 
to cross-market others products and services. Consider a 
consumer who purchases a pair of golf shoes. The box is 
Bedoop encoded. By presenting the box to a Bedoop system, 60 
the consumer is linked to a web page that presents various 
promotional offers. The consumer may, for example, elect to 
play a free roimd of golf at one or more identified local golf 
courses, or print a coupon for ten percent off any order of 
socks from an on-fine sock merchant. (Various means can be 65 
employed to prevent multiple redemptions from a single 
box. One is a serial number that is tracked by the web page 
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or cross-marketed merchant, and only honored once. 
Another is identification data corresponding to the consumer 
that is tracked to prevent multiple redemptions.) 

Product tags can likewise be Bedoop-encoded. A tag from 
an article of Nike apparel can lead to the Nike on-line store, 
where the user can buy more merchandise. If the tag is from 
a soccer jersey, a certain tag manipulation (e.g., rotate left) 
may lead the user to a special-interest soccer page, such as 
for the World Cup. A tag on a golf glove may lead to a 
website of a local golf course. Twist left to reserve a tee time; 
twist right to review course maps and statistics. Bedoop 
kiosks can be provided in retail stores to let consumers use 
the Bedoop features. 
Travel Planning Services 

After making a reservation at a resort, a consumer is 
typically mailed (by email or conventional mail) various 
confirmation information. If not already printed, the con- 
sumer can print this information (e.g., a confirmation card). 

Bedoop-encoding on the printed object can lead to web- 
based information relating to the reservation (e.g., reserva- 
tion number, the consumer's name, arrival/departure dates, 
etc.). If the consumer wishes to make dinner or goff 
reservations, this object is presented to a Bedoop system — 
either at the user's home, at an airport kiosk, etc. The system 
recognizes the object type and encoded data, and establishes 
a link to a remote computer that provides various informa- 
tion and scheduling services for the resort. By manipulating 
the object (or otherwise) the consumer selects desired dinner 
and golf tee times. The system already has the reservation 
number (indexed by the UID), so tedious provision of such 
data is avoided. 

In some embodiments, the remote computer is not main- 
tained by the resort, but is rather maintained by an indepen- 
dent travel service. (The travel service may also maintain the 
DNS leaf node server.) The computer can present a web page 
(branded by the travel service or not) that offers the sched- 
uling options desired by the user, and also presents links to 
other information and services (e.g., offering entry tickets to 
nearby attractions, and advertising nearby restaurants). 

Airline tickets (or e-ticket confirmations) can be similarly 
encoded with Bedoop data. These items may be presented to 
Bedoop systems — at a traveler's home or in airports — to 
permit review and changing of travel itinerary, reserve hotels 
and rental cars, secure first-class upgrades, check the air- 
plane's seating arrangement, review frequent flier status, 
scan tourist information for the destination, etc. 
Movie Tickets 

As indicated earlier, movie tickets can be encoded with 
Bedoop data identifying, e.g., the movie title and date. When 
a movie viewer returns home, the ticket stub can be pre- 
sented to a Bedoop system. One of the options presented by 
the corresponding Bedoop application can be to launch a 
pay-per-view screening of the just-seen movie at a dis- 
counted rate. Another is to download the movie onto a 
writeable DVD disk at the viewer's home, perhaps serialized 
to permit playback only on that viewer's DVD player, or 
enabled for only a few playbacks, etc. (again, likely for a 
discounted fee). Still another option is to present web- 
delivered video clips from the movie. Another is to offer 
related merchandise for purchase, possibly at discount to 
retail. (These features may be available for only a limited 
period after the date encoded on the ticket stub.) Another is 
to alert the consumer to upcoming movies of the same 
genres, or with the same director or stars, or released by the 
same studio. Still another is to direct a web browser to an 
on-line ticket merchant for tickets to other movies. The 
consumer may navigate among these options by manipulat- 
ing the ticket stub, or otherwise. 
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The same, or related, options can likewise be provided in 
response to Bedoop data detected from a book jacket pre- 
sented to a Bedoop system. 
Video Recording 

A video recording device can be programmed to record a 5 
broadcast program by presenting a Bedoop sensor with a 
printed promotion for the program (e.g., an advertisement in 
a newspaper or TV Guide). Bedoop-encoded within the 
printed document is data by which the Bedoop system 
(which may be built into the video recorder or separate) can lO 
set the recording time, date, and channel. 
Set Top Boxes 

Many entertainment-related applications of Bedoop data 
can be implemented using television set top boxes. Such 
boxes include processors, and typically include a return is 
channel to a control facility. The provision of a Bedoop chip 
and optical sensor can vastly increase the functionality these 
devices presently provide. 
Special Event Tickets 

Consider a ticket to a basketball game. By presenting the 20 
ticket to a Bedoop system, a user may access the web site of 
either team so as to review recent scores and statistics. The 
user may also obtain a web-based virtual tour of the arena, 
and review seating maps. Tickets for upcoming games may 
be ordered, as well as pay-per-view games and team sou- 25 
venirs. For high-priced tickets, the user may be entitled to 
premium web features, such as on-line text-, audio-, or 
video-chat session with a team star on the day before the 
game. 

Unlike conventional tickets, Bedoop-encoded tickets 30 
need not limit the user to a predetermined seat. While the 
ticket may be printed with a nominal seat, the user may 
present the ticket to a Bedoop sensor and access a web site 
at which a different seal can be reserved. On attending the 
event, the consumer presents the ticket to a Bedoop sensor 35 
that reads the ticket UID and looks up the seat assignment 
most-recently picked by the consumer. It then prints a chit 
entitling the consumer to take the seat earlier selected from 
the transactional web site. 

Signet Rings 40 

Signet rings have historically been used to indicate a 
person's identity or office. Such rings, or other items of 
personal jewelry, can be encoded with Bedoop data (either 
by texturing or printing) and presented as necessary to 
Bedoop systems. The extracted Bedoop data can lead to a 45 
secure web site indicating the person's name and other 
information (i.e., a web site that has anti-hacking measures 
to prevent illicit change of the stored identification 
information). Such a signet ring can be presented to Bedoop 
systems that require a high-confidence confirmation of 50 
identity/authorization before proceeding with a Bedoop 
function. 
Post-It® Notes 

Pads of Post-It® notes, or other pads of paper, can be 
marked by the manufacturer (either by texturing, water- 55 
marked tinting, ink-jet spattering, etc.) to convey stegano- 
graphic data (e.g., Bedoop data). When such a note is 
presented to a Bedoop system, the system may launch an 
application that stores a snapshot of the note. More 
particularly, the application may mask the note-portion of 60 
the image data from the other image data, virtually re-map 
it to a rectangular format of standardized pixel dimensions, 
JPEG-compress the resulting image, and store it in a par- 
ticular computer subdirectory with a name indicating the 
date of image acquisition, together with the color and/or size 65 
of the note. (These latter two data may be indicated by data 
included in the Bedoop payload.) If the color of the note is 
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indicated by digital data (e.g., in the file name), then the 
image itself may be stored in grey-scale. When later recalled 
for display, the white image background can be flooded with 
color in accordance with the digital color data. 

The Bedoop system may buffer several past firames of 
image data. When the object is recognized as a Post -It note 
whose image is to be saved, the system may analyze several 
such frames to identify the one best-suited for storage (e.g., 
check the spatial frequency content of the note as imaged in 
each frame, to identify the one with the finest detail), and 
store that one. 

When a Post-It note is recognized by the Bedoop system, 
the system may emit a confirmation tone (or other response) 
to indicate that the object has been recognized, but not 
immediately execute the snapshot operation. Instead, the 
system may await a further instruction (e.g., gesture) to 
indicate what operation is desired. 

By moving the note towards the sensor, for example, the 
user can signal that a snapshot operation is to be performed. 
(This closer presentation of the note may also permit the 
imaging system to capture a more detailed frame of image 
data.) 

By moving the note away, the system may respond by 
reading, decompressing, and displaying the six most- 
recently stored Post-It note images, in tiled fashion, on the 
computer screen. The individual notes can be displayed at 
their original dimensions, or each can be re-sized to fill the 
full height or width of a tile. A user interface control 
(responsive to gestures, mouse operation, keyboard scroll 
arrows, etc.) allows the user to scroll back in time to any 
desired date. 

The full 64-bit Bedoop payload of other embodiments 
may not be needed for Post-It notes. In the just-given 
example, for example, the Bedoop system responds to all 
Post-It notes in the same fashion. Thus, an abbreviated 
Bedoop format that indicates simply *rm a Post-It note, 
yellow, size 3"x3'" can sufi&ce. The twelve bit CLASS ID, 
with eight further bits to indicate color/size combinations, 
may be sufficient. Reducing the payload permits it to be 
more robustly encoded on small objects. (As noted below, 
Bedoop decoding systems can look for several different data 
formats/protocols in trying to extract Bedoop data from an 
object.) 

Ahgnment of Documents for Other Purposes 

While the just-described pre-marked paper triggered a 
Bedoop response when presented to a Bedoop sensor (i.e., 
take a snapshot of the paper), the markings can be used for 
ptuposes other than to trigger Bedoop responses. Regardless 
of the particular data with which the paper is encoded, the 
embedded subliminal graticules, or other 
steganographically-encoded registration data, can be used 
by other applications to correct misafignment of scanned 
data. In a photocopier, for example, a document need not be 
placed exactly squarely on the glass platen in order to yield 
a properly-aligned photocopy. The scanner scans the skewed 
document and then detects the steganographic registration 
markings in the resulting scan data. This data is then 
processed to virtually re-register same, so that the registra- 
tion markings are in a desired alignment. The processed scan 
data is then provided to the xerographic reproduction unit to 
yield a photocopy in which the skew effect is removed. 

The same technique is likewise applicable to video 
recorders, digital cameras, etc. If such a device images an 
object (e.g., a photograph) with steganographic registration 
markings, these markings can be used as a guide in 
re-registering the resulting data to remove mis-alignment 
effects. 



07/08/2004, EAST Version: 1.4.1 



us 6,31i;214 Bl 

31 32 

Postal Mail Information More on Optical Input Devices 

Many contexts arise in which data to be presented to a It is expected that image input devices will soon become 
consumer is valuable only if timely. The postal service mail commonplace. The provision of digital cameras as built-in 
is ill-suited for some such information due to the latency components of certain computers (e.g., the Sony Vaio 
between printing a document, and its ultimate delivery to a 5 laptops) is just one manifestation of this trend. Another is 
recipient. Bedoop principles, however, allow the recipient to camera-on-a-chip systems, as typified by U.S. Pat. No. 
take a postal object that was printed well before delivery, 5,841,126 and detailed in Nixon et al.. "256x256 CMOS 
and use it on receipt (i.e., present to a Bedoop system) to j^^^^^ pi^d Sensor Camera-on-a-Chip," IEEE J. Solid- 
receive up-to-the-minute informaUon. In this and other ^^^^ Circuits, Vol. 31(12), pp. 2046-2051 (1996), and 
embodiments the Bedoop data can also uniquely identify ..^^^^ ^ Electronic Camera-on-a- 
the addre^ee/recipient/user, so the web site can present data ^^^.p „ j^^^ TransacUons of Electron Devices, vol. 44, No, 
custoriii^d to mat user. - „ „ 10, October 1997. Still another is head-mounted cameras (as 

Distributors of printed advertismg can reward Bedoop- ' 1 , * . . j • • 

driven consumer v^its to their web sites by issuing digital P^^f J" computer-augmented vision 

tokens or coupons that can be redeemed for premiums, systems) These and other image input devices are all 

cash-back. etc. Every mniionth visitor wins a mUlion pen- ^5 suitable for use in Bedoop systems, 

nies (with appropriate safeguards, e.g., preventing more than Camera-on-a-chip systems can be equipped with Bedoop 

one entry an hour). detector hardware integrated on the same chip substrate. 

Classes of Bedoop Encoding This hardware can be arranged to find and decode Bedoop 

The above-described embodiments focused on use of data fi"om the image data — notwithstanding scale, rotation, 

Bedoop data after decoding. Additional insight may be 20 differential scaling, etc. Gestural decoding can also be 

gained by examining the earlier part of the process- provided in hardware, with the resulting data output in 

encoding. packet form on a serial output bus. Such a chip can thus 

Encoding can be performed in many contexts, which may provide several outputs — image data (either in raw pixel 

be conceptualized as falling into three broad classes. The form, or in a data stream representing the image in one of 

first is static marking, in which a document designer, pre- 25 various image formats), 64 bits of Bedoop data (serially or 

press service bureau, advertising agency or the like embeds ^ paraUel), and decoded gesture data. 

Bedoop data. The second is dynamic marking, in which ^j^er embodiments, the Bedoop detector (and/or the 

automated systems encode, or vary, Bedoop data ';on die ^^^^^^ decoder) can be on a substrate separate from the 

fly." Such systems can tailor the Bedoop data to particularly camera svstem 

suit the context, e.g., to the moment, place, user, etc. The accommodate different Bedoop data formats and 

P^tocols, the hardware can include R;^ or ROM in which 

to a document at the time 01 pnntmg. ' . ^ , . - . . ^ , 

TTie second class of encodiig enables features not avail- ^^^'^^^ format/protocol information is stored (Hiese dif- 

able from the first. Consider an American Express travel web f^^^* formal^protocols can relate, e.g to Bedoop systems 

page with information about travel to Hawaii. A DNS leaf employing differem data payload lengths, different sublum- 

node server points to this page in response to certain Bedoop 35 nal grids, different encoding techniques, etc.) As the Bedoop 

data — e.g., data encoded in a magazine photograph of a system stares out and grabs/analyzes frames, each frame can 

Hawaiian beach scene. be analyzed in accordance with several different formats/ 

Actually, all Bedoop data having a certain CLASS and protocols to try and find a format/protocol that yields valid 

DNS ID may lead to this web page, irrespective of the UID Bedoop output data, 

data. If the magazine photo is encoded with a particular 40 Movable Bedoop Sensors 

"don't care" UID field(e.g., 111111111111111111111111), Although the iUustrated Bedoop systems are generally 

this may signal the originating Bedoop system — or any stationary, they need not be so. They can be portable. Some 

intervening system through which the Bedoop data passes — such systems, for example, employ palmtop computers 

that arbitrary data can be inserted in the UID field of that equipped with optical sensor arrays. If the palmtop is 

Bedoop packet. The originating Bedoop system, for 45 provided with live network connectivity (e.g., by wireless), 

example, can insert a dynamically-configured series of bits then Bedoop applications that rely on remote computers can 

into this field. Some of these bits can provide a profile of the be implemented just as described. If the palmtop is not 

user to the remote server, so that the Bedoop response can equipped with live network coimectivity, any Bedoop appli- 

be customized to the user. (The user would naturally pre- cations that rely on remote computers can simply queue such 

approve information for such use so as to allay privacy 50 communications, and dispatch same when the palmtop next 

concerns.) has remote access (e.g., when the palmtop is next placed in 

As one example, the local Bedoop system can set the least its recharger and is coupled to a modem through which 

significant bit of the UID field to a "0" if the user is male, internet access can be established). 

or to a "1" if the user is female. The next four bits can Another variant is a Bedoop sensor that is movable 

indicate the user's age by one of sixteen age ranges (e.g., 3 55 around a desk or other work-surface, like a mouse. Such a 

or less, 4-5, 6-7, 8-9, 10-11, 12-13, 14-15, 16-17, 18-20, sensor can be coupled to the associated computer by cabling, 

21-24, etc.). or a wireless interface can be used. The peripheral may be 

Alternatively, or in addition, the local Bedoop system can arranged for placement on top of an item in order to read 

stuff the don't-care UID field (all of it, or in part) with digital data with which the object is marked. (Built-in 

signature data tending to uniquely identify the local Bedoop 60 illumination may be needed, since the device would likely 

system (e.g., system serial number, a hash code based on shadow the encoding.) Some forms of such peripherals are 

unchanging data unique to that system, etc.) By reference to adapted to serve both as general purpose digital cameras, 

such data, the remote server can identify repeat visits by the and also as Bedoop sensors. 

same user, and can tailor its responses accordingly (e.g., by Such a peripheral would find many applications. In "read- 
recalling a profile of information earlier entered by the user 65 ing" a magazine or book, for example, it may be more 
and stored at the remote server, avoiding the need for data intuitive to place a Bedoop reader "on" the object being 
re-entry). read, rather than holding the object in the air, in front of a 
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Bedoop sensor. This is particularly useful, e.g., when a Conclusion 

magazine page or the like may have several differently- Having described and illustrated the principles of our 

encoded Bedoop sections (corresponding to different invention with reference to illustrative embodiments, it 

articles, advertisements, etc.), and the user wants to assure should be recognized that the invention is not so limited, 

that the desired Bedoop<ncoded section is read. 5 For example, while certain of the embodiments were 

™. 1 1 » ^- f • . t u,„„,^^«. u» illustrated with reference to internet-based systems, the 

The "bookmark paradigm of mtemet browsers might be \ • -i i r ui * 

supplemented with V bookmark, e.g Bedoop data rorpui:^^^^^^^^^ bS 

encoded on one or more pages of paper. To direct a browser ^J^^ ^^^^ ^^^^^ ^^^^ Compuserve, dial-up 

to a particular bookmarked destination, the penpheral is ^^^^ ^ ^^^^ Likewise, for internet-based 

simply placed on top of the page (or part thereoQ that is lO embodiments, the use of web browsers and web pages is not 

marked with the correspondmg Bedoop data. A user may essential; other digital navigation devices and other on-line 

print a "Map" comprised of postage stamp-sized regions jj^ta repositories can be similarly accessed, 

tiled together, each of which regions represents a favorite Similarly, while the details of the preferred Bedoop sys- 

web destination. tem were particularly given, the underlying principles can be 

Such a map may be printed on a mouse pad. Indeed, ^5 employed in numerous other forms, 

mouse pads with certain maps pre-encoded thereon may be For example, one other form is to steganographically 

suitable as promotional materials. A company may offer to encode physical objects with Digital Object Identifiers 

print a family photograph on such a pad. Encoded within the (DOIs). The Center for National Research Initiatives and the 

photograph or the pad texture are addresses of web sites that Digital Object Identifier Foundation (www.doi.org) have 

have paid a fee to be accessible in this manner on a user*s 20 performed extensive work in estabUshing an infrastructure 

jjegj^ by which digital objects can be distributed, tracked, and 

Like mice— which are provided with buttons, roller managed. Some of this same infrastructure arid technology 

wheels, and roller buttons in addition to X-Y encoders ^e adapted, m accordance with the teachmgs provided 

movable Bedoop encoders can likewise be provided with ^^»>ove, to associate new functionality with physical objects, 

auxiliary switches and roUer inputs to complemem the data 25 Another form is not to reference a remote data repository 

input provided by the optical sensor. Indeed, some embodi- ^^ta embedded on an object, but instead to encode the 

ments integrate the functions of Bedoop peripheral with a ^^^^^^^ data directly on the object. A photograph, for 

mouse. Cnie undersides of mice are generally under-utUized, ^'^^"^P^^' ^'^^'^^y ^"^/^^ "^^^ ^ telephone number, 

and can readily be equipped with an image sensor.) Gestural presentmg the photograph to an optical sensor on the 

input can readUy be provided by such a peripheral-in this telephone, the telephone can analyze the optical information 

context moving the sensor rather than the object. to extract the telephone number and dia the number^. 

Watermarkine Techniques without the need for any external data, Similariy, a pnnted 

„ . . . r 1. . 1 office document (e.g., spreadsheet) can be encoded with the 

•niere are nearly as many techniques for digital water- ^^^^ corresponding electronic file, 
marking (steganographic data encoding) as there are apph- 35 ^^viating the need for indirect linking (e.g., to a database to 
cations for it. TTie reader is presumed to be famjhar with the ^ID to a computer address). Most of the above- 
great variety of methods. A few are reviewed below. ^^^^^ embodiments are suitable for such direct encoding 

The present assignee s pnor appucation Sen No. 09/127, related data 
502, filed Jul. 31, 1998, shows techniques by which very fine business card example given above, the detailed 
hnes can be prmted on a medium to slightly change the ^ techniques can be supplementary to existing optical charac- 
medium's apparerit tmt while also conveying digital data recogniUon techniques. That is, the image data from an 
Commonly-owned applicaUon Ser. No. 09/074,034, filed ^-^^ ^e appUed both to a Bedoop decoder and 
May 6, 1998, details how the contours of printed imagery ^ q^R system. Text characters discerned by the OCR 
can be adjusted to convey digital data. (That techmque can ^ ^^^^^^ ^.^^^^j -^^^ ^ ^^^^^^^ ^^^^ger 
be applied to prmted text charactere^ as weU as the hne art database. The techniques employed in ±e Bedoop 
imagery parliculariy considered.) l^e assignee s U.S. Pat. ^ y^^^^^ ^^^^^ object and handle visual 
No. 5,850,481 details how the surface of paper or other distortion (e.g., the visual artifacts due to scale, rotation, 
media can be textured to convey optically-detectable bmary . advantageously be used in OCR detection as weU, 
data. The assignee's U.S. Pat. No. 5.841,886 and 5,809,160 permitting extraction of the OCR information without care- 
detail various techniques for steganographically encoding ^ placement of the card. 

photographs and other imagery. y^^^^ ^^^^^^ of foregoing embodiments made refer- 
Some watermarking techniques are based on changes ^uce to ink-jet printing, similar advantages can often be 
made in the spatial domain; others are based on changes obtained with other printing technologies, e.g., laser/ 
made in transformed domains (e.g., DCT, wavelet). Water- xerographic printing, offset printing, etc. 
marking of printed text can be achieved by sUght variations 55 foregoing embodiments, Bedoop decoding gener- 
to character shape, character kerning, line spacing, etc. ally proceeded from image data obtained from a physical 
Data glyph technology, as detailed in various patents to object. However, in some contexts, it is advantageous to 
Xerox, is usable in many of the applications detailed herein. Bedoop-decode image data provided electronically, e.g.. 

The foregoing is just a gross under-sampfing of the large over the internet, 

number of watermarking techniques. The artisan is pre- 60 Likewise, while the foregoing embodiments generally 

sumed to be familiar with such art, all of which is generally reUed on Bedoop image sensors that stared out for an object 

suitable for use in the applications detailed herein. at an expected point, in alternative embodiments, sensors 

More generally, essentially any data encoding method that that seek rather than stare can be employed (as was illus- 

permits recovery of the encoded data from optical scan data trated above in connection with the elevator example), 

can be employed. Bar codes (ID and 2D) are but the most 65 Similarly, while the illustrated embodiments generally 

familiar of many such optically-detectable data encoding employed sensors that repeatedly grabbed frames of image 

techniques. data, this need not be the case. Single frame systems, such 
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as flatbed scanners, and video systems arranged to grab 
single frames — with or without TWAIN interfaces — can 
alternatively be used. 

As indicated above, while steganographic encoding of the 
digital data is used in the preferred embodiments, visible 
forms of digital encoding — such as bar codes — can naturally 
be employed where aesthetic considerations permit. 

In certain of the embodiments, digital data conveyed by 
means other than optical can be tised. Electromagnetic 
detection (e.g., of the sort used in proximity-based card- 
access systems) can be arranged to decode digital data, 
permitting "at-a-distance" reading of data from physical 
objects, just as in the foregoing embodiments. 

Since the Bedoop image sensors typically acquire plural 
frames of data, the extraction of the digital data can be based 
on more than a single image frame. More confidence in the 
results may be accumulating decoded data over several 
frames. Moreover, movement of the object within the sen- 
sor's field of view may permit the system to acquire infor- 
mation from other perspectives, etc., enhancing system 
operation. 

While the preferred embodiments employ 2-D image 
sensors (e.g., CCDs), other optical sensing technology can 
alternatively be employed. Supermarket laser scanners, for 
example, can read bar-code data, Raster-scaiming of such 
systems can permit acquisition of 2-D data (either in bit- 
mapped form, or grey-scale). 

While the illustrated embodiments used a 12/24/24 bit 
protocol for CLASS/DNS/UID data, other arrangements can 
of course be used. In some applications it is advantageous 
for the protocol to more nearly match those commonly used 
for internet communications. For example, IP addresses for 
internet Domain Name Servers (DNS) are presently 32 bits, 
with extension to 64 or 128 bits foreseen in the near future. 
The DNS field in Bedoop systems can be follow the internet 
standard. 

Some embodiments can advantageously employ texture- 
based Bedoop encoding of objects. Bedoop texturing can be 
effected by various means, including pressure rollers, chemi- 
cal or laser etching, etc. 

While the foregoing embodiments have generally 
employed planar objects to convey the digital encoding, this 
need not be the case. Objects of other shapes can likewise be 
employed. Some shapes present relatively straightforward 
image processing tasks. Data imaged from a soft drink can 
or other cylindrical surface, for example, is fairly easy to 
remap using known geometrical transforms so as to essen- 
tially "unwrap" the printing from the can. Other geometries 
can present more complex re-mappings, but are likewise 
generally within the capabilities of the artisan. (Such remap- 
ping is facilitated by encoding in the data certain reference 
markings, such as subliminal graticules, etc. The unknown 
3D shape of the object being imaged can usuaDy be inferred 
from the apparent warping of the reference markings in the 
2D image data generated by the scanner. Once the warping 
is characterized, it is generally straightforward to un-warp so 
as to prepare the image data for decoding.) 

It was once popular to predict that paper documents 
would be replaced with electronic media. In hindsight, 
electronic media may be recognized as a poor surrogate for 
paper. Electronic media conveys information flawlessly, but 
is lacking in experiential attributes. We can hold paper, stack 
it, own it, deface it, give it, guard it, etc. It provides an 
opportunity for physical dominion entirely lacking with 
electronic media. 

From the foregoing discussion it can be seen that, rather 
than replacing paper with electronic media, perhaps the 
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future lies in giving paper digital attributes — hybridizing the 
physical experience of paper with the technical advantages 
of digital media. Such an arrangement makes available a 
great wealth of new functionality, now accessible through 
5 familiar paper items, rather than through a "computer input 
peripheral." 

To provide a comprehensive disclosure without unduly 
lengthening this specification, applicant incorporates by 
reference the patents, applications, and publications identi- 
10 fied above. 

In view of the many embodiments to which the principles 
of my invention may be applied, it should be recognized that 
the detailed embodiments are illustrative only and should 
not be taken as limiting the scope of my invention. Rather, 
15 I claim as my invention all such embodiments as fall within 
the scope and spirit of the following claims, and equivalents 
thereto. 

APPENDIX A 

20 PAPER-BASED CONTROL OF COMPUTER 

SYSTEMS 

Related Application Data 

This application is a continuation-in-part of co-pending 
25 applicaaon Ser. No. 09/130,624, filed Aug. 6, 1998, which 
is a continuation of application Ser. No. 08/508,083 (now 
U.S. Pat. No. 5,841,978). 

The subject matter of this application is generally related 
to that in aU of the assignee's other patents and applications, 
30 e.g., U.S. Pat. Nos. 5,841,886, 5,832,119, 5,822,446 and 
5,841,978, and the application entitled Methods and Sys- 
tems Employing Digital Watermarking, filed on even date 
herewith. 

35 Field of the Invention 

The present invention relates to use of printed documents 
to control computer systems. Exemplary documents include 
business cards, advertisements, and identification badges, 
but the invention is not so limited. 

^ Background and Summary of the Invention 

Over the past century, business cards have fonned part of 
business ritual. Fimctionally, they serve as a record of an 
encounter, and detail means by which the giver may be 

45 contacted (address, phone, etc.). 

Business cards have changed, essentially, not at all in 
response to the advent of computers. Some accommodation 
has been made for business cards on the computer side, in 
the form of specialized scanner and optical character rec- 

50 ognition tools by which textual data printed on cards can be 
read and entered into personal productivity software tools 
(e.g. contact managers, address books, datebooks, personal 
information managers, etc.). However, the data transferred 
into the personal productivity software is static and 

55 unchanging. 

In accordance with one embodiment of the present 
invention, the textual information on a business card is 
supplemented with steganographically-encoded, muhi-bit 
binary data. This latter data does not significantly distract 

60 from the visual aesthetics of the card (as would a bar code 
or the like), yet can be used by an associated computer to 
initiate a link to an internet site corresponding to the 
business card giver. At the site, the recipient of the card may 
gain access to the giver's schedule, and other information 

65 that changes over time. (Such information may not generally 
be available over the internet to persons without the card 
data.) 
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The foregoing and additional features and advantages of 
the present invention will be more readily apparent from the 
following detailed description, which proceeds with refer- 
ence to the accompanying drawings. 

Brief Description of the Drawings 

FIG. 1 (omitted) shows a flow chart of a process accord- 
ing to one embodiment of the present invention. 

Detailed Description 

Digital watermarking is a quickly-growing field of 
endeavor, and many techniques are known. Generally, all 
seek to steganographically convey multi-bit data ancillary to 
some other signal or medium. 

The present assignee's prior application Ser. No. 09/127, 
502, filed Jul, 31, 1998, shows techniques by which very fine 
lines can be printed on a medium to slightly change the 
medium's apparent tint, while also conveying digital data. 
Commonly-owned application Ser. No. 09/074,034, filed 
May 6, 1998, details how the contours of printed imagery 
can be adjusted to convey digital data. (That technique can 
be applied to printed text characters, as well as the line art 
imagery particularly considered.) Applicant's U.S. Pat. No. 
5,850,481 details how the surface of paper or other media 
can be textured to convey optically-detectable binary data, 
/^plicant's U.S. Pat. Nos, 5,841,886, 5,809,160, and the 
priority applications detailed above, detail various tech- 
niques for steganographically encoding photographs and 
other imagery. 

Three papers by Brassil et al show other techniques for 
conveying watermark data by slight changes to printed text, 
"Electronic Marking and Identification Techniques to Dis- 
courage Document Copying," Proceedings of INFOCOM 
'94 Conference on Computer, IEEE Comm. Soc 
Conference, Jun. 12-16, 1994, pp. 1278-1287; "Hiding 
Information in Document Images," November, 1995, 7 
pages, AT&T Bell Laboratories Technical Report (available 
at ciss95_ps.z from ftp://ftp.research.att.com/dist/brassil/ 
1995/ciss95.ps.Z.; and "Document Marking and Identifica- 
tion using Both Line and Word Shifting," INFOCOM '95 
(available at ftp: //f tp.rese arch. att,com/dist/brassil/l 995 
infocom95.ps.Z). 

The foregoing is just a sampling of the large literature on 
watermarking. The artisan is presumed to be familiar with 
such art, all of which is generally suitable for use with the 
novel concepts detailed below. 

In accordance with any of the known watermarking 
techniques, a biisiness card is steganographically encoded 
with plural bit data. At least part of this data identifies an 
internet address or web site at which data about the giver of 
the card is stored. If suflBcient bits can be encoded into the 
business card, the address can be encoded literally, e.g., by 
ASCII or binary numeric encoding. Alternatively, to reduce 
the data payload, an abbreviated form of address. One 
example of such an abbreviated form is a Unique Identifier 
(UID) which can be, e.g., a 24-bit value. 

Desirably, the steganographic encoding is tailored to 
facilitate decoding in the presence of arbitrary rotation or 
scale distortion of the card introduced during scanning, 
(Some such techniques are shown, e.g., in applicant's related 
patents identified above. Others are known to artisans.) 

As shown in FIG. 1 (omitted in this appendix), the card is 
scanned (e.g. by use of conventional op to-electronic 
devices, such as a scarmer or a digital camera). 'Rie output 
data is then optionally processed to account for any skew or 
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scale factor. The plural-bit digital daU is then decoded and 
stored, eg., in personal productivity software. 

(Although not particularly shown in FIG. 1, it is expected 
that the detailed process will often be supplemental to 

5 known OCR-reading of business cards, and entry of the 
textual data into personal productivity software. That is, the 
scan data is processed both by OCR techniques, and by 
steganographic decoding techniques, and the results of both 
operations stored in a data structure or other memory for 

10 later reference.) 

The steganographically-decoded plural-bit data is pro- 
vided to a web browser or other internet appliance and used 
to initiate a link to a remote computer over the internet's 
network of computers. If the remote address was literally 

^5 encoded in the business card, that address is used directly. If 
an abbreviated form of address was encoded, an additional 
step may be required. 

If a UID was encoded in the card, rather than a literal 
address, the web browser might consult an index to correlate 
the UID to an address. The index could be a table or other 
data structure stored on the user's local computer, but more 
commonly is a remote name server database to which the 
browser links as a default when processing business card 
UIDs. Data obtained from the index is then used to complete 
the linking to the ultimate destination. (In addition to reduc- 
ing the business card payload, such linking through an 
index, e.g., by a UID, offers flexibility in that the ultimate 
destination can be moved to other server sites as needed, 
with just a simple update to the index. Alternatively, all 

■'^ business cards encoded with the former address would be 
rendered obsolete if the site were relocated.) 

At the ultimate site, the user is presented with whatever 
information the business card giver chooses to provide, 

22 including biographical information, photos, promotional 
offers or advertisements relating to the card-giver's business 
(or relating to enterprises to whom the card-giver has rented 
screen space), etc., etc.. In one embodiment, the giver's site 
is linked to the giver's personal productivity tool(s) and 
permits viewing, e.g., of calendar information (showing 
where the business card giver is scheduled to be today, or for 
the rest of the week, month, etc.). 

Typically, this calendar information is not available to 
casual web browsers; the steganographically decoded data 

45 from the business card includes some authentication data 
(akin to a password) that permits access to otherwise 
restricted data. This authentication data can take the form of 
a web page address to which no publicly-accessible link 
points, a password that is separately presented to the web 

50 server by the user's browser after a link is established, or 
other known technique. 

In one form of the invention, the giver of business cards 
may have several differently-encoded cards, each with a 
different level of access authorization. Thus, some cards may 

55 access a biographical page without any calendar 
information, other cards may access the same or different 
page with access enabled to today's calendar, and still other 
cards may access the same or different page with access 
enabled for the card-giver's complete calendar. 

60 The reference to business cards and personal calendars is 
illustrative only. The invention is more widely applicable. 
Going back a century, "calling cards" were used by persons 
whose interests were strictly social, rather than business. 
The principles of the present invention can similarly be 

65 applied. Teenagers can carry small cards that can be 
exchanged with new acquaintances to grant access to private 
dossiers of personal information, favorite music, artwork 
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video clips, etc. The cards can be decorated with art or other 256 CMOS Active Pixel Sensor Camera -on-a-Chip," IEEE 
indicia that can serve purposes wholly unrelated to the J. Solid-State Circuits, Vol. 31(12), pp. 2046-2051 (1996), 
linkiDg data steganographically encoded therein. and Possum, "CMOS Image Sensors: Electronic Camera- 
Even the "card" paradigm is too restrictive. The same on-a-Chip," IEEE Transactions of Electron Devices, vol. 44, 
techniques can be applied to any object. A music CD cover 5 No. 10, October 1997. Still another is head-mounted cam- 
can be encoded to point to a promotional site associated with eras (as are presendy used in some computer-augmented 
the music artist. A book jacket can link to a similar site. vision systems). These and other image input devices can all 
Printed advertising distributed through the US mail (cards, be used in connection with the present invention, 
magazines, etc.) can be encoded to point to related web- To facilitate embodiments of the present invention, a prior 
based promotional sites. (Sponsors of such advertising or art camera-on-a-chip system can be modified to also include 
other sites can reward visits to their internet site by issuing a steganographic watermark detector on the same semicon- 
visitors digital tokens or coupons that can be redeemed for ductor substrate. Such a chip — ^in addition to providing 
premiums, cash-back, etc., either for any such visit, or only image output data — can also analyze the image data to 
if the visit was effected through the portal of a discern any steganographically encoded data, and produce 
steganographically-encoded printed medium.) corresponding output data. (Again, such analysis desirably 
Many contexts arise in which data to be presented to a includes correction for scale and rotation factors, so precise 
consumer is valuable only if timely. The postal service maU positioning of the object being "read" is not essential for 
is ill-suited for some such information due to the latency correct decoding.) 

between printing a document, and its ultimate dehvery to a To provide a comprehensive disclosure without unduly 

recipient. The principles of the present invention allow the lengthening this specification, applicants incorporate by 

recipient to take a steganographically-encoded data object reference the patents, applications, and publications identi- 

(card, etc.) that was printed well before delivery, and use it fied above, 
on receipt to receive up-to-the-minute information, (In this 

and other embodiments, the steganographically-encoded APPENDIX B 
data can also include data uniquely identifying the recipient/ 
user, so the web site can present data customized to that 
user.) 

The present technology also has application in access Field of the Invention 

control systems. An identification badge (either with photo ^ . , ... ^ • , 
or graphics, or with text alone) can be encoded with stega- '° Th^ present mvention relates to apphcations of digital 

nographically access control data (e.g.. access codes or watennarkmg m conjunction with audio, video, unagery. 

digital keys) that is recognized by optical-scanner-equipped ^^^^^ content, 

locks and the like, permitting access by authorized persons Background 
to restricted areas or restricted services (e.g., computer 

privileges). Given the low cost of media and printing (as Watermarking (or "digital watermarking") is a quickly 

compared with other access control technologies), the cards growing field of endeavor, with several different approaches, 

can be issued on a daily, weekly, or other firequent interval. The present assignee's work is reflected in U.S. Pat. Nos. 

and the access control system can be programmed to pennit 5,841,978, 5,768,426, 5,748,783, 5,748,763, 5,745,604, 

access in response to such cards only for the pre-set limited 5,710,834, 5,636,292, 5,721,788, and laid-open PCT appli- 

period. Lost cards soon lose their threat. cations W097/43736 and WO99/10837. Other work is illus- 

Tickets to sporting events, concerts, and other events can Crated by U.S. Pat. Nos. 5,734,752, 5,646,997, 5,659,726, 

be steganographically encoded to permit the bearer to access 5,664,018, 5,671,277, 5,687,191, 5,687,236, 5,689,587, 

premium web content available only to those who have 5,568,570, 5,572,247, 5,574,962, 5,579,124, 5,581,500, 
purchased tickets (e.g., an on-line text-, audio-, or video-chat 45 5,613,004, 5,629,770, 5,461,426, 5,743,631, 5,488,664, 

session with the featured performer or sports star the day 5,530,759, 5,539,735, 4,943,973, 5,337^61, 5,404,160, 

before the event). Alternatively, the encoded data may link 5,404,377, 5,315,098, 5,319,735, 5,337,362, 4,972,471, 

to a transactional site. In some such embodiments, the ticket 5,161,210, 5,243,423, 5,091,966, 5,113,437, 4,939,515, 

is printed with a nominal show data and seat assignment, but 5,374,976, 4,855,827, 4,876,617, 4,939,515, 4,963.998, 
also includes a UID in addition to the encoded address of an 50 4,969,041, and published foreign Wlicat^^^^^^^ 

associated transactional ticket site. The user then can visit 98/02864, HP 822,550, WO 97/39410, WO 96/36163, GB 

the transactional web site to change seating (or date). On 2,196.167. EP 777.197, EP 736,860. EP 705.025, EP 766. 

attending the event, the consumer presents the ticket to a ^68, EP 782,322 WO 95/20291, WO 96/26494, WO 

steganographic decoder apparatus that discerns the UID and 96/36935. WO 96/42151, WO 97/22206, WO 97/26733. 
looks up the seat assignment most-recently picked by the 55 Most of the work in watermarking, however, is not in the 

consumer. It then prints a chit entitling the consumer to take patent literature but rather in published research. In addition 

the seat earlier selected on-line. to the patentees of the foregoing patents, some of the other 

The reference to "scanning" of objects naturally brings to ^^^^ers in this field (whose watermark-related writings can 

mind images of desktop flatbed scanners, or multi-function found by an author search in the INSPEC database) 
hydra devices. While such devices can be used— together 60 1°^!^^^ I. Pitas, Eckhard Koch, Jian Zhao, Norishige 

with convention digital cameras (including video Morimoto, Laurence Boney, Kineo Matsm, A. Z. Tirkel, 

cameras)— the inventors foresee that image input devices ^^^^ ^intzer, B. Macq, Ahmed H. Tewfik, Frederic Jordan, 

will soon be much more commonplace. The provision of Naohisa Komatsu, and Lawrence O'Goraian. 

digital cameras as built-in components of certain computers The artisan is assumed to be familiar with the foregoing 
(e.g., the Sony Vaio laptops) is just one manifestation of this 65 P^or art. 

trend. Another is camera-on-a-chip systems, as typified by In the present disclosure it should be understood that 

U.S. Pat. No. 5,841,126 and detailed in Nixon et al., "256x references to watermarking encompass not only the assign- 
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ee's watermarking technology, but can likewise be practiced 
with any other watermarking technology, such as those 
indicated above. 

Watermarking has various uses, but the present specifi- 
cation details several new uses that provide functionality and 5 
features not previously available. 

Brief Description of the Drawings (omitted ficom 
this >^pendix) 

FIG. 1 is a diagram showing the participants, and 
channels, involved in the distribution of music. 

FIG. 2 shows a conceptual model of how music artists, 
record labels, and E-Music distributors can all interact with 
a Media Asset Management System, of which several are -jj 
detailed in the following specification. 

Detailed Description 

For expository convenience, much of the following dis- 
cussion focuses on miisic, but the same principles and 20 
techniques are largely or wholly applicable to other source 
data, whether non-music audio, video, still imagery, printed 
materials, etc. 

Music Asset Management 

Referring to the figures, the music distribution process 
begins with a creative artist 10. The artist's music has 
traditionally been distributed by a record label 12. (While 
the following discussion refers to distribution through such 
a label, it should be understood that such distribution can 

• 30 

just as well be effected directed under the artist's control, 
without a record label intermediary.) 

In traditional distribution 14, the record label produces 
tangible media, such as records, tapes, videos (eg. music 
videos), and CDs 16. These media are physically distributed 
to end-consumers 18. Additionally, the label 12 distributes 
the music media to outlets 20, such as radio and TV stations, 
cable and satellite systems, etc., which broadcast (or 
narrowcast) the artist's work to an audience. Distribution 
through such media outlets may be monitored by playout ^ 
tracking services. Playout tracking data, collected by firms 
including Arbitron, Nielsen, ASCAP, BMI, etc., can be used 
to compute royalty payments, to verify broadcast (e.g. for 
advertising), etc. 

Increasingly, the distribution of the music to the media 
outlets is performed electronically. Such distribution first 
took the form of analog audio over high quality landHnes or 
satellite channels. Digital audio quickly supplanted analog 
audio in such distribution channels due to higher fidelity. 

More recently, distribution of the music from the record 50 
labels to the media outlets has occurred over secure links, 
now including the internet. Such security was first provided 
simply by scrambling the audio signal or data. More sophis- 
ticated "container**-based systems are now coming into 
vogue, in which the audio is "packaged" (often in encrypted 55 
form) with ancillary data. 

Electronic distribution of music to the consumer is also 
gaining popularity, presently in the MP3 format primarily. 
The music providers may deal directly with the public, but 
more commonly effect such consumer distribution through a 60 
newly emerging tier of digital media outlets, such as internet 
sites that specialize in music. From such sites, consumers 
can download digital audio files into personal digital audio 
players. (The Diamond Rio, and the Audible MobilePlayer 
devices are some of the first of what will doubtless be a large 65 
number of entrants into this personal internet audio appli- 
ance market.) Or the downloaded data can be stored by the 



consumer-recipient onto any other writeable media (e.g. 
hard disk, CD, DVD, tape, videotape, etc.). Typically a 
personal computer is used for such downloading, but this 
intermediary may be dispensed with by coupling next gen- 
eration of personal audio appliances to an internet-like link. 

The data downloaded by the consumer can be stored 
either in the native digital format, translated into another 
digital format (which translation may include decryption), 
converted into analog and recorded in analog form, etc. 

Unauthorized copying or use of the music can occur 
anywhere in the foregoing channels. However, one of the 
greatest risks occurs once the music has been delivered to 
the consumer (whether by tangible media, by traditional 
broadcast media outlets, by emerging digital distribution, or 
otherwise). 

The general idea of embedding auxiliary data into music 
(i.e. watermarking) has been widely proposed, but so far has 
been of limited applicability. 

For example, GoodNoise is planning to embed a digital 
signature — termed a multimedia identifier, or MMI — in its 
MP3 music. MMI will register the song and its author with 
a licensing number. In addition to providing information 
about the songwriter and distributor, this digital encoding 
may also include lyrics, liner notes, and other information. 
But all of the proposed uses serve only to convey informa- 
tion from the distributor to the consumer; use for "tracking" 
is actively disclaimed. (Wired News, "GoodNoise Tags MP3 
FUes," Feb. 3, 1999.) 

The Genuine Music Coalition — a partnership of various 
companies in the music distribution business — likewise has 
aimounced plans to employ watermarking of MP3 music. 
The watermarking technology, to be provided by Liquid 
Audio, will convey data specifying the artist or producer 
contact, copyright data, and a niunber to track ownership. 
The Coalition hopes that the provision of this embedded 
information will help thwart piracy. Industry observers 
believe Liquid Audio will next introduce playback technol- 
ogy only plays audio in which its watermark is detected. 
(Wired News, "Liquefying MP3," Jan. 23, 1999.) 

A similar initiative has been aimounced by the Recording 
Industry Association of America (RIAA). Termed the Secure 
Digital Music Initiative (SDMI), the program seeks to define 
a voluntary specification that will assure proper compensa- 
tion to those who produce and distribute music. One element 
of the system will likely be a watermarking component. 
(Dow Jones Newswire, "Spurred By Maverick Technology, 
Music Industry Eyes Web," Dec. 31, 1998.) 

Yet another initiative has been announced by Solana and 
ASCAP. Other companies promoting watermarking for 
music include Aris Technology, MCY.com, and AudioSoft. 

The watermark payload can represent various types of 
data. An exemplary payload includes data relating to the 
artist, distribution entity, title, and copyright date/proprietor. 
Additionally, the payload can include a digital object 
identifier — an ISBN-like number issued by a central orga- 
nization (e.g. a rights management organization) to uniquely 
identify the work. 

Such payload data can be encoded literally (e.g. the title 
by a series of ASCII characters, etc). In other embodiments, 
codes or abbreviations can be employed — with each code 
having a known meaning. In still other embodiments, the 
data can be meaningless by itself, but may serve as a key 
(e.g., a Unique Identifier, or UID) into a remote data 
database or repository. An example of such a remote data 
repository is a web site at a Master Global Address (MGA) 
associated with content, as detailed below. 
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An exemplary data payload may, for example, have the take various forms, e.g. a synchronization signal, a marker 
following format: signal, calibration signal, a universal code signal as 

described in applicant's patents, etc.) 



A 


B 


C 


D 


E 


F 


G 


H 1 I 1 


1 



In some embodiments, a watermark "dial-tone" signal is 
provided. This dial-tone signal is a low amplitude, relatively 
wideband, repetitive signal that commonly conveys only 
limited information (e.g. a single bit of information). Its 

Where A is a six-byte (8-bits to a byte) ASCII string serving presence in an audio signal can serve as a "do not record," 

as a digital object identifier (which may serve as a link to a or similar instruction signal. Alternatively, or in addition, the 

Master Global Address through a default name server, as dial-tone signal can serve as an aid in "locking" to a 

discussed below), B is a two-byte ASCII field serving as a ^0 plural-bit digital watermark signal that is also encoded in the 

key into an "artist" field of the remote database, C is a audio. For example, the cyclical repetition of the signal can 

three-byte ASCII field serving as a key into a "title" field of identify the start of the plural-bit digital watermark 

the remote database; D is a 14-bit field serving as a key into signal. Or the spectrum or repetition rate of the signal can 

a "label" field of the remote database, E is an 8-bit integer identify any temporal corruption of the audio. An exemplary 

representing the work's year of first publication (with 0 15 g^ch signal is detailed as a "simple universal code" in U.S. 

representing the year 2000); F is a 10-bit field serving as a Pat. No. 5,636,292. 

key into a "price" field of the remote database, G is a a track of music can be pre-authorized for specified types 
two-byte usage control string (detailed below), H is a of use. For example, the usage control string of the water- 
streaming data channel, and I is a string of bits serving as a mark payload may include a six-bit field detailing the classes 
cyclic redundancy checksum for the foregoing. (More 20 of devices for which the audio is authorized. Each bit would 
sophisticated error correcting checksums can, of course, be correspond to a different class of device. Class 1 devices 
employed.) This payload format totals 136 bits, exclusive of may be personal playback devices with only analog-audio 
the CRC coding and the streaming data channel. output. Class 2 devices may be personal entertainment 

This payload is encoded repeatedly, or redundantly devices capable of outputting music in digital (e.g. MP3, 

through the music, so that the full payload can be decoded 25 redbook, *. WAV) format, as well as analog audio. Class 3 

from partial excerpts of the music. devices may be personal computer systems (i.e. with essen- 

The encoding is also desirably perceptually adaptive, so tially unlimited ability for processing and outputting digital 

that higher energy encoding is employed where the listener audio). Etc., etc. A device to which such MP3 audio is 

is less likely to perceive the additional "noise" introduced by provided would check the usage control string data to 

the encoding, and vice versa. Various techniques for per- 30 determine whether it is authorized to utilize the audio. A 

ceptually adaptive encoding are known. For example, some personal playback device with analog-only output, for 

tie the amplitude of the encoded signal to the instantaneous example, would examine the first bit of the usage control 

amplitude of the music. Others exploit psychoacoustic string. If it was "1," the device would be authorized to use 

"masking'* of one signal by a spectrally-or temporally- (i.e. playback) the MP3 data; if it was a "0," the device 

adjoining signal of higher energy. Still other approaches fill 35 would refuse to play the music. 

gaps in the music's spectrum with watermark energy. These In addition to pre-authorization for certain classes of 

and other techniques are detailed in the patents incorporated devices, the u^age control string can also include bits 

by reference. indicating the number of permitted playbacks. This data can 

In other embodiments, perceptually adaptive encoding is be encoded in bits seven through nine, representing eight 

not used. In some such embodiments, no tailoring of the 40 possibiUties: 

temporal or spectral characteristics of the watermark signal o — no playback permitted 

is employed. In others, the watermark signal is spectrally 1— single playback permitted 

filtered to emphasize low firequency audio components (e.g. j-two playbacks permitted 

less than 500 ^) high frequency audio components (e.g. ^^^^^ playbacks permitted 

higher than 2500 hz), or mid-frequency audio components 45 . . i i_ i j 

(500-2500 hz). 4-four playbacks permitted 

The streaming data field channel (H) is a medium by 5— five playbacks permitted 

which data can be conveyed from a distribution site (or other 6—10 playbacks permitted 

site) to the end user. Such data may be entirely unrelated to 7-^inlimited playbacks permitted 

the underlying work. For example, it may serve a utilitarian so 8 — refer to associated data (within the watermark, or 

purpose, such as conveying data to a memory in the con- stored at a remote site) which specifies number of 

sumer device to replace previously-stored data that is out- permitted playbacks. 

of-date. It may be a commercial channel on which band- The playback device may include a non-volatile store in 

width is sold for access to the consumer or the consumer's which the number of permitted playbacks is stored for each 

device. Essentially any purpose can be served by this 55 track of music. The device would decrement this number at 

streaming data field. Unlike most of the other fields, the the beginning of each playback. 

streaming data field may not endlessly repeat the same data. The usage control string can also include a two-bit field 

but can convey data that changes with time. (bits ten and eleven) indicating recording permissions. A 

Desirably, the encoding is performed in a manner permit- value of 0 means that data corresponding to the MP3 audio 

ting recovery of the watermark data even if the audio is 60 (regardlessof digital format) should never be made available 

corrupted, e.g. by format conversion, re-sampling, tape wow to another digital device. A value of 1 means that the data 

and flutter, compression, coding, or various forms of audio corresponding to the MP3 data may be made available once 

processing (e.g. filtering, pre-emphasis, re-scaling, etc.). to another digital device, A value of 2 means that the data 

One way to provide for such robustness is to encode a signal may be made available an unlimited number of times to 

of known character thai can be recognized through all such 65 other digital devices. (Value 3 is reserved.) 

corruption. By identifying such known signal, the water- Another data field that can be included in an audio 

mark signal can then be decoded. (The known signal can watermark is a rating that indicates age-appropriateness. 
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Music with violence or sexual themes might be given a The combined cassette/radio player has now evolved into 

rating akin to the MPAA "PG-13" or "R" rating. Audio a general purpose computer with wide-ranging functionality, 

appliances may be programmed to recognize the rating of and other sophisticated devices. Music can be acquired off 

incoming mtisic, and to interrupt playback if the rating the web, and can be recorded in various forms (e.g. in a 

exceeds a certain threshold setting. Various known tech- 5 personal MP3 player, stored on a hard disk, stored on a 

niques can be employed to assure that such settings cannot writeablc CD-ROM, played back and recorded on analog 

readily be changed, e.g., by juvenile listeners. cassette, etc., etc.). The quality can be quite high, and the 

Another data field that can be included in an audio ^^^^^^^ broadcast time problems of radio broadcasts have 

watermark is a date field. This field can mdicate either the overcome by the web's on-demand delivery mecha- 

datc the music was watermarked or a date in the future on (Moreover, the music can be downloaded in faster- 

which certain rights associated with the music shou d ^^an-realtime, a further benefit over recording-off-the-air 

change. Some consumers, tor example may not wish * u • \ 

purchase perpetual playback rights to certain musical selec- ^ t . - ■, . . j 1 j - 1 j - / 

fions. TTie ri^t to play a selection for 6 months may suffice ^ ^ne hybnd between the new and old is a novel radio (e.g. 

for many consumers, especially if the price is discounted in ^ ^0 that has a capture button on the front panel 

view of the limited term. Such an arrangement would not be ^5 (or other form of user mterface, e.g., a Capture icon on a 

whoUy disadvantageous to music distributors, since some GUI). If a user hears a song they want to record and keep, 

consumers may end up purchasing music twice if their initial they press the Capture button while the song is playmg. In 

assessment of a musical selection's appeal was too short- response, the radio device decodes a watermark embedded 

sighted. (Naturally, the playback equipment would require a in the music, and thereby knows the identity of the music, 

source of real-time clock data against which the date field in 20 The radio then makes a wireless transmission identifying the 

the watermark can be checked to ensure that the playback user and the desired song. A local repeater network picks up 

rights have not yet expired.) the wireless signal and relays it (e.g. by wireless rebroadcast. 

Another of the data fields that can be included in an audio by modem, or other communication medium) to a music 

watermark specifies technical playback parameters. For clearinghouse. The clearinghouse charges the user a nominal 

example, the parameter can cause the playback appliance to 25 fee (e.g. via a pre-arranged credit card), and queues the 

apply a spectral equalization that favors bass frequencies, or music for download to a predetermined location associated 

treble frequencies, or mid-range frequencies, etc. Other with the user. 

pre-configured equalization arrangements can similarly be In one embodiment, the predetermined location is the 

invoked responsive to watermark data. Likewise, the param- user's own computer. If a "live" IP address is known for the 

eter can invoke special-effects provided by the playback 30 user's computer, the music can be transfened immediately, 

appliance, e.g., echo effects, reverb, etc. (Again, such If the user's computer is only occasionally connected to the 

parameters are usually represented in an abbreviated, coded internet, the music can be stored at a web site (e.g. protected 

form, and are interpreted in accordance with instructions with a user-set password), and can be downloaded to the 

stored in a memory (either in the playback appliance, or user's computer whenever it is convenient, 

linked thereto). 35 In other embodiments, the predetermined location is a 

The same data fields and principles can be applied to personal music Ubrary maintained by the user. The library 

non-audio content. In video, for example, watermarked data can take the form, e.g., of a hard-disk or semiconductor 

can adaptively control the display monitor or playback memory array in which the user customarily stores music, 

parameters (e.g., color space) to enhance the viewing expe- This storage device is adapted to provide music data to one 

rience. 40 or more playback units employed by the user (e.g. a personal 

Music Asset Management/Commerce MP3 player, a home stereo system, a car stereo system, etc.). 

The majority of domestic music piracy is not organized. In most installations, the library is physically located at the 

Rather, it is a crime of opportunity and convenience. If the user's residence, but could be remotely sited, e.g. consoli- 

crime were made more difficult, the alternative of obtaining dated with the music libraries of many other users at a 

a copy through legitimate channels would be less onerous. 45 central location. 

Similarly, if the procedure for obtaining a copy through The personal music library can have its own internet 
legitimate channels were simplified, the incentive for piracy connection. Or it can be equipped with wireless capabilities, 
would be reduced. Watermarking facilitates both — making permitting it to receive digital music from wireless broad- 
the crime more difficult, and making legitimate mxisic acqui- casts (e.g. firom the clearinghouse). In either case, the library 
sition easier. 50 can provide music to the user's playback devices by short- 
Consider, for example, the pricing of music in conven- range wireless broadcast, 
tional record stores. A CD (compact disk) may cost $15, but By such arrangement, a user can conveniendy compile an 
its sale may be driven by just one or two popular songs on archive of favorite music — even while away from home, 
the disk. To obtain these songs, the consumers must pur- Many variants ofthe foregoing are of course possible. The 
chase the entire disk, with perhaps a dozen songs of no 55 radio can be a portable unit (e.g. a boombox, a Walkman 
particular interest. This, in essence, is a tying arrangement radio, etc.), rather than an automotive unit. The UI feature 
that benefits the record labels while prejudicing the consum- employed by the user to initiate capture a musical selection 
ers. Given these circumstances, and a ready opportunity to need not be a button (physical or on-screen). For example, 
make copies, it is not surprising that customers sometimes in some embodiments it can be a voice-recognition system 
make illicit copies. 60 that responds to spoken commands, such as "capture" or 

One classic technique of avoiding purchase of a complete "record." Or it can be a form of gesture interface, 

collection of music, when only one or two songs is desired. Instead of decoding the watermark only in response to the 

is to record the music off the radio. While of dubious user's "capture" command, the radio can decode watermarks 

legality, this technique was popular in the era of combined from all received programs, and keep the most recent in a 

cassette/radio players. However, the desired music was 65 small FIFO memory. By such arrangement, the user need not 

sometimes difficult to encounter in a radio broadcast, and the issue the capture instruction while the song is playing, but 

quality was less than superb. can do so even after the song is finished. 
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In some embodiments, data corresponding to the water- identify CDs on which the then-playing selection is 
mark can be made available to the user in various forms. For recorded. Or the listener may query the appliance for the 
example, it can be presented to the user on an LCD screen, then-playing artist's concert schedule. Again, the appliance 
identifying the artist and song currently playing. If a corre- can contact a remote database, relay the query, and forward 
spending UI button is activated, the device can so-identify 5 data from the watermark payload identifying the artist 
the last several selections. Moreover, the data need not be and/or song tide to which the query relates. The database 
presented to the user in displayed form; it can be annunci- locates the requested data, and relays same back to the 
ated by known computer-speech technologies instead, appliance for presentation (via a display, by machine speech. 
In embodiments in which the watermark does not convey ^^^^j. ^^^^j ^ ^^^^ desired, the user can continue 
ASCII text data, but mstead conveys UIDs, or coded the dialog with a forther instruction, e.g., to buy one of the 
abbreviations, the device must generaUy mterpre this data ^^^^ then-playing song is included. Again, this 
before presenting it to the user. In an illustrative . ^ ^. , . j u • r^m * j 
embodiment, the device is a pocket-sized FM radio and is "^^^^^ "^f^ !!^^"^"^ J^^' . ' ' I u 
equipped with a 1 megabyte semiconductor non-volative P^^'^^hed from the applicance to the clearinghouse which can 
RAM memory. The memory includes a data structure that ^^en complete the transaction in accordance with pre-stored 
serves as a look-up table, matching code numbers to artist ^5 information (e.g. credit card account number, maihng 
names and song titles. When the user queries the device to address, etc.). A confirming message is relayed to the 
learn the identify of a song, the memory is indexed in appliance for presentation to the user, 
accordance with one or more fields from the decoded While the foregoing transactions require a link to a remote 
watermark, and the resulting textual data from the memory site or database, other watermark-based consumer services 
(e.g. song title and artist) is annunciated or displayed to the 20 can be provided vidthout such a link. For example, a user can 
user. query the appliance as to the artist or song-title of the 
In most applications, such memory will require frequent selection currently playing. The appliance can consult the 
updating. The RF receiver provides a ready mechanism for embedded watermark data (and optionally consult a memory 
providing such updated data. In one embodiment, the radio to determine the textual names associated with coded water- 
" awakens" briefly at otherwise idle moments and tunes to a 25 mark data), and provide the requested information to the 
predetermined frequency at which updated data for the user (e.g., by a display, annunciation, or other output), 
memory is broadcast, either in a baseband broadcast The foregoing concepts (e.g. Capture, Send, etc.) can also 
channel, or in an ancillary (e.g. SCA) channel. be employed in connection with internet- rather than radio- 
In variants of the foregoing, internet delivery of updated delivery of music. (The following discussion is illustrated 
memory data can be substituted for wireless delivery. For 30 with reference to the "Capture" function, but it will be 
example, the artist/song title memory in the personal player recognized that the other earlier-discussed features can be 
can be updated by placing the player in a "nest" every similarly implemented.) 

evening. The nest (which may be integrated with a battery There are many commercial web sites that sell audio (in 
charger for the appliance) can have an internet connection, CD form or otherwise), and offer Hmited free music 
and can exchange data with the personal device by infrared, 35 downloads, (or music clips) as an enticement to lure con- 
inductive, or other proximity-coupling technologies, or sumers. But there are also a great number of music web sites 
through metal contacts. Each evening, the nest can receive that have no commercial pretense. They are hosted by music 
an updated collection of artists/song titles, and can re-write lovers strictly for the enjoyment of other music lovers. When 
the memory in the personal device accordingly. By such music is downloaded from such a web site, the end-user's 
arrangement, the watermark data can always be properly 40 computer can analyze the digital data to decode watermark 
intepreted for presentation to the user. data therefrom. Again, the user can be presented with a 

TTie "Capture" concepts noted above can be extended to "Capture" button that initiates a commercial transaction, by 
other functions as well. One is akin to forwarding of email. which a complete copy of the then-downloaded audio is sent 
If a consumer hears a song that another friend would enjoy, to a prearranged storage location, and the user's credit card 
the listener can send a copy of the song to the friend. This 45 is debited accordingly. This transaction can occur indepen- 
instruction can be issued by pressing a "Send" button, or by dently of the site from which the music is downloaded (e.g. 
invoking a similar function on a graphical (or voice- or through the clearinghouse referenced above), 
gesture-responsive) user interface. In response, the appli- While the "Capture" button can be presented on the 
ance so-instructed can query the person as to the recipient. web-site, this would generally not be in keeping with the 
The person can designate the desired recipient(s) by typing 50 non-commercial nature of such web sites. Instead, in an 
in a name, or a portion thereof sufiBcient to uniquely identify exemplary embodiment, the Capture feature is a software 
the recipient. Or more typicaUy, the person can speak the program resident at the user's computer. When this software 
recipient's name. As is conventional with hands-free vehicle program is invoked by the user, a socket channel is instan- 
cell phones, a voice recognition unit can listen to the spoken tiated between the user's computer and the clearinghouse 
instructions and identify the desired recipient. An "address 55 over the then-existing internet connection. The decoded 
book" -like feature has the requisite information for the watermark data and user ID is transmitted to the clearing- 
recipient (e.g., the web site, IP address, or other data house over this channel, without interrupting the user's other 
identifying the location to which music for that recipient activity (e.g. downloading music from the non-commercial 
should stored or queued, the format in which the music web site). In response, the clearinghouse transmits the music 
should be delivered, etc.) stored therein. In response to such 60 to the prearranged location and attends to billing, 
command, the appliance dispatches instructions to the In some embodiments, a watermark detector is included 
clearinghouse, including an authorization to debit the send- as part of the operating system, and constantly monitors all 
er's credit card for the music charge. Again, the clearing- TCP/IP, or other internet, data received by the user's 
house attends to delivery of the music in a desired manner computer, for the presence of watermarks. In such case, 
to the specified recipient. 65 when the Capture feature is invoked, the program examines 

Still further, a listener may query the appliance (by voice, a memory location in which the operating system stores the 

GUI or physical button, textual, gesture, or other input) to most- recently received watermark data. In another 
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embodiment, the computer does not monitor all internet 
traffic for embedded watermark data, but includes an API 
that can be called by the Capture program to decode a 
watermark from the data then being received. The API 
returns the decoded watermark data to the Capture program, 5 
which relays same to the clearinghouse, as above. In still 
another embodiment, the watermark decoder forms part of 
the Capture program, which both decodes the watermark 
and relays it to the clearinghouse when the Capture program 
is invoked by the user. 

There are various techniques by which the Capture pro- 
gram can be selectively invoked. One is by a keyboard 
macro (e.g. by a combination of keyboard keys). Another is 
by a program icon that is always presented on the screen, and 
can be double-clicked to activate. (Again, confirmation 
processes may be called for, depending on the likelihood of 
inadvertent invocation.) Many other techniques are likewise 
possible. 

In the just-contemplated scenario, the Capture operation 
is invoked while the user is downloading music from a 
non-commercial web site. This seems somewhat redundant, 20 
since the downloading — itself — is transferring music to the 
user's computer. However, the Capture operation provides 
added value. 

In the case of streaming audio, the audio is not typically 
stored in a location in which it can be re-used by the 25 
consumer. It can be listened-to as delivered, but is then gone. 
Capturing the audio provides the user a copy that can be 
played repeatedly. 

In the case of downloaded music files, the music may 
have been encoded to prevent its recordal on other devices. 30 
Thus, while the user may download the music onto a desktop 
computer, copy-prevention mechanisms may prevent use of 
that file anywhere else, e.g. on a portable music appliance. 
Again, Capturing the audio provides the user a copy that can 
be transferred to another device. (The music file provided by 35 
the clearinghouse can have copy-prevention limits of its 
own — e.g., the file can be copied, but only once, or the file 
can be copied only onto devices owned by the user.) 

(Confirmation of device ownership can be implemented in 
various ways. One is to identify to the clearinghouse all 40 
music devices owned by a user at the time the user registers 
with the clearinghouse (supplemented as necessary by later 
equipment acquisitions). Device IDs associated with a user 
can be stored in a database at the clearinghouse, and these 
can be encoded into the downloaded music as permitted 45 
devices to which the file can be copied, or on which it can 
be played.) 

The commerce opportunity presented by non-commercial 
music web-sites is but one enabled by digital watermarks. 
There are many others. 50 

To take one example, consider the media by which music 
and artists are presently promoted. In addition to radio 
airtime, these include music videos (a la MTV), fan 
magazines, web advertisements, graphical icons (e.g. the 
Grateful Dead dancing bears), posters, live events, movies, 55 
etc. Watermarked data can be used in all such media as a link 
in a commercial transaction. 

A poster, for example, typically includes a photo of the 
artist, and may comprise cover-art from a CD. The photo/art 
can be digitally watermarked with various types of data, e.g., 60 
the artist's identify, the record label that distributes the 
artist's work, the music project being particularly promoted 
by the poster (e.g. a CD, or a concert tour), a fan web-site 
related to the artist, a web-site hosted by the record label for 
selling audio in CD or electronic form, a web-site from 65 
which free music by the artist can be downloaded, data 
identifying the poster itself, etc. 
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A user, equipped with a portable appliance that merges the 
functions of palmtop computer and digital camera, can snap 
an image of the poster. The processor can decode the 
watermarked data, and initiate any of various links based on 
the decoded data. 

In an exemplary embodiment, after snapping the picture, 
the user invokes a software program on the device that 
exposes the various links gleaned from the snapped image 
data. Such a program can, for example, present the option of 
linking to the artist's fan web site, or downloading free 
streaming audio or music clips, or ordering the promoted 
CD, or requesting the above-noted clearinghouse to down- 
load a personal copy of selected song(s) by the artist to the 
user's personal music library, etc. (The device is presumed 
to have a wireless internet link. In devices not having this 
capability, the requested actions can be queued and auto- 
matically executed when a link to the internet is available.) 

Still more complex transactions can be realized with the 
use of a remote database indexed by digital watermark fields 
decoded from the poster. For example, the poster may 
promote a concert tour. Fields of the digital watermark may 
identify the artist (by a code or full text), and a web site or 
IP address. The user appliance establishes a link to the 
specified site, and provides the artist identifier. In response, 
the site downloads the tour schedule for that artist, for 
display on the device. Additionally, the downloaded/ 
displayed information can include a telephone number that 
can be used to order tickets or, more directly, can indicate the 
class of seats still available at each (or a selected) venue, and 
solicit a ticket order from the user over the device. The user 
can supply requested information (e.g. mailing address and 
charge card number) over the return channel link (wireless 
or wired, as the case may be), and the ticket(s) will be 
dispatched to the user. In the case of a wireless link all of this 
can occur while the user is standing in front of the movie 
poster. 

Similar systems can be implemented based on watermark 
data encoded in any other promotional media. Consider 
music videos. Using known TV/computer appliances, water- 
mark data added to such videos can readily be decoded, and 
used to estabhsh links to audio download, CD-sales, fan 
club, concert ticket outlet web sites, etc., as above. 

Even live events offer such watermark-based opportuni- 
ties. The analog audio fed to public address or concert 
speakers can be watermarked (typically before 
amplification) to encode plural-bit digital data therein. A 
next generation personal music appliance (e.g. one with a 
wireless interface to the internet) can include analog record 
capability (e.g. a built-in microphone, analog-to-digital 
converter, MP3 encoder, coupled to the unit's semiconduc- 
tor memory), A user who attends a live event may record an 
excerpt of the music. The watermark can then be decoded, 
and the extracted data used to access the links and commerce 
opportunities reviewed above. 

Cinema movies offer both audio and visual opportunities 
for watermark-based commerce opportunities. Either 
medium can be encoded to convey information of the types 
reviewed above. A personal appliance with image- or audio- 
capture capabilities can capture an excerpt of the audio or 
imagery, decode the watermark data therefrom, and perform 
any of the linking, etc., functions reviewed above. 

The consumer-interest watermarks reviewed above are 
only exemplary. Many others will be recognized as useful. 
For example, promotional clips presented before a feature 
film presentation can include watermark data that point 
(either by a literally encoded web address link, or by an ID 
code that indexes a literal hnk in a remote link database) to 
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reviewer critiques of the previewed movies. Watermark data More particularly considering the "copy once" usage 

in a featured film presentation can lead to web sites with restriction, an illustrative embodiment provides two distinct 

information about the movie stars, the director, the producer, watermark payload bits: a "copy once" bit and a "copy 

and can list other movies by each of these persons. Other never" bit. When originally distributed (whether by intemet, 

watermark-conveyed web links can present opportunities to 5 wireless, or otherwise), the "copy once" bit is set, and the 

buy the movie on videotape, to purchase the movie "copy never" bit is un-set. 

soundtrack, to buy movie-related toys and games, etc. When music encoded in this fashion is provided to a 

More on Device Control compliant recording device, the device is authorized to make 

Much of the foregomg has focused on watermark encod- compUant device is one that recognizes 

ing to provide enhanced customer experiences or opportu- ^^^^^^^ watermark data, and behaves as dictated by the 

nities. Naturally, watermarks data can alternatively, or ^.^^^j,.) When this privilege is exercised, the recording 

additionally, serve the interests of the media owner. ^^^^ ^^^^^ | 

To illustrate, consider watermarked music. The media . 1-1 t *u n * * ^ u j- . *u- 1* «• ■ 

owner would be best served if the watermark serves dual ^ P°^*^*^- illustrated embodiment, this alteration is 

purposes: permissive and restrictive. Permissively, music ^^^''^^ ^y the recording device adding a second watermark 

appliances can be designed to play (or record) only music ^5 to both the music, with the "copy never^ bit asserted. The 

that includes an embedded watermark signaUng that such second watermark must generaUy be encoded in an 

activity is authorized. By this arrangement, if music is "orthogonal" domain, so that it will be detectable notwith- 

obtained from an unauthorized source and does not include standing the continued presence of the original watermark, 

the necessary watermark, the appliance will recognize that it Compliant equipment must then check for both watermarks, 

does not have permission to use the music, so will refuse 20 and refuse to copy if either is found to have the "copy never" 

requests to play (or record). bit asserted. 

As noted, music appliances can respond restrictively to One advantage to this arrangement is that if the water- 
the embedded watermark data to set limits on use of the mark signal has undergone some form of corruption (e.g. 
music. Fields in the watermark can specify any or all of (or scaling or resampling), the first watermark may have been 
others in addition to) (a) the types of devices on which the 25 weakened. In contrast, the second watermark will be native 
music can be played (b) the types of devices on which the corrupted signal, and thus be more easily detected, 
music can be recorded; (c) the number of times the music corruption may also contribute to the orthogonality of 
can beplayed; (d) the number of times the music can be watermark relative to the other, since the two water- 
recorded, etc. marks may not have precisely the same time base or other 

The device restrictions (a) and (b) can be of various types. foundation ) 
In some embodiments, the restrictions can identify particular a 1. 1. • ^ *u « _»» 
units (e.g. by serial nilmber, registered owner, etc.) that are ^ "^^"^^'^^ approach is no to encode the copy never* 
authorized to play/record the encoded music. Or the restric- \° ^^e onginal music, but leave this bit (m whatever 
tions can identify particular classes of units (e.g., battery- mamfestation) blank (i.e. neither 1 nor (f ). In transform- 
powered portable players with music memories of less than based watermark techniques, this can mean leavmg trans- 
50 megabytes, disk-based dedicated music appliances, gen- 35 form coefl5cient(s) correspondmg to the "copy never** bit 
eral purpose personal computers, etc.). Or the restrictions un-changed. If the watermarking is effected in the temporal 
can identify particular performance quality criteria (e.g., two sample domain (or spatial domain, for image data), this can 
channel, 16-bit audio at 44.1 KHz sample rate, or lower mean leaving certain samples pixels) unmodified. The 
quality). recording device can then alter the transform coefficients 
The use restrictions (c) and (d) can likewise be of various 40 and/or samples as necessary to assert the previously- 
types. Examples include "do not copy," "copy once only,'* unencoded "copy never" bit when the permitted recording is 
"unrestricted copying permitted,** "play once,*' "play N made. 

times" (where N is a parameter specified elsewhere in the In such a system, compliant recording devices check for 

watermarked data, or by reference to a database indexed by the "copy never** bit in the sole watermark, and refuse to 

a watermark data field), "unrestricted playing permitted," 45 make a copy if it is asserted (ignoring the value of any "copy 

etc. once" bit). 

It is straight forward to design a music appliance to A third approach to "copy once** is to set both the "copy 

respond to usage limits of zero (e.g. "do not copy**) and once'* and "copy never*' bits, but set the former bit very 

infinity (e.g. "unrestricted copying permitted," and "unre- weakly (e.g. using lower gain and/or high frequency DCT 

stricted playing permitted"). The device simply examines 50 coeflBcients that do not survive certain processing). The frail 

one or more bits in the watermark data, and permits (or "copy once" bit is designed not to survive common 

refuses) an operation based on the value thereof. corruptions, e.g., resampling scaling, digital to analog 

Implementation of the other usage-control restrictions can conversion, etc. To further assure that the "copy once" bit is 

proceed in various ways. Generally speaking, the stored lost, the recording device can deliberately add a weak noise 

music can be altered to give effect to the usage-control 55 signal that masks this bit (e.g. by adding a noise signal in the 

restrictions. For example, if the music is "record-once,** then firequency band whose DCT coefficient conveys the "copy 

at the time of recording, the appliance can alter the music in once" bit). In contrast, the "never copy" bit is unchanged and 

a fashion indicating that it now has "do not record" status. reliably detectable. 

This alteration can be done, e.g., by changing the watermark In such a system, compliant devices check for the "copy 

data embedded in the stored music (or adding watermark 60 once" bit in the sole watermark, and refuse to make a copy 

data), by changing other data stored in association with the if it is not detected as set. 

music, etc. If the original signal is stored (as opposed, e.g., These three examples are but illustrations of many pos- 

to a streaming signal, such as an internet or wireless sible techniques for changing the rights associated with a 

transmssion), it too should be so-altered. work. Many other techniques are known. See, e.g., the 

Likewise with playback limitations. Tlie number of play- 65 proposals for watermark-based copy control systems for 

backs remaining can, e.g., be encoded in an updated water- digital video at the Copy Protection Technical Working 

mark in the music, be tracked in a separate counter, etc. Group, http://www.dvcc.com/dhsg/, fi-om which certain of 
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the foregoing examples are drawn. See also Bloom et al, Pinnacle Systems, SoundForge, Sonic Foundry, Xing 

"Copy Protection for DVD Video," IEEE Proceedings, Technology, Prosoniq, and Sonic Desktop Software. 

Special Issue on Identification and Protection of Multimedia Payment-Based Systems 

Information, June, 1999. Another aspect of the present invention is the use of 

Scaleability 5 anonymous payment tokens that can be used to obtain 

One feature that is desirable in many detector embodi- content on the web. In one embodiment, a token comprises 

ments is scaleability. This refers to the ability of a detector a 128-bit pseudo-random number, to which additional bits 

to scale its computational demands to match the computa- identifying an issuing bank (or other issuing institution) are 

tional resources available to it. If a detector is running on a appended. (The additional bits can be the IP address of a web 

high performance Pentium III workstation, it should be lO server of the bank, a routing number identifying the bank for 

"doing more" than if the same detector is running on a slow electronic wire transfers, or other identifier.) The 128-bit 

microcontroller. One way scalability can be achieved is by numbers are randomly generated by the bank — commonly 

processing more or less chunks of input data (e.g. temporal as needed — ^and each represents a fixed increment of money, 

excerpts of music, or blocks/macroblocks of pixels in a e.g. ten cents. 

frame of video data) to decode watermarks. For example, an 15 A consumer wishing to have a store of currency for such 

input audio stream might be broken into chunks of one commerce pays the bank, e.g., $10 in exchange for 100 

second each. A fast processor may complete decoding of tokens. These tokens are transferred electronically to disk or 

each chunk in less than a second, permitting it successively other storage in the consumer's computer in response, e.g., 

to process each chunk in the data stream. In contrast, a slow to a credit card authorization, or may be provided by diskette 

processor may require two and a half seconds to decode the 20 or other storage medium over the counter at a bank branch 

watermark firom a chunk. While it is processing a first chunk, (in which case the consiuner thereafter copies the numbers 

the second and third pass by un-decoded. The processor next into storage of his or her computer). (Outlets other than 

grabs and processes the fourth chunk, permitting the fifth banks can of course be employed for distributing such 

and sixth to pass by un-encoded. numbers, much in the manner that convenience and many 

The detector running on the fast processor is clearly more 25 grocery stores commonly issue money orders.) 

difficult to "fool," and yields a decoded watermark of higher Imagine that the consumer wishes to view the final quarter 

confidence. But both systems decode the watermark, and of a Trailblazer basketball game that aired on television a 

both operate in "real time." week ago. (The consumer may have either missed the game. 

The skipping of input data in the temporal (e.g. music or or may have seen it but wants to see the last quarter again.) 

video) or spatial (e.g. image or video) domain is but one 30 The user directs a web browser to a web site maintained for 

example of how scaleability can be achieved. Many other such purpose and performs a search to identify the desired 

approaches are known to those skilled in the art. Some of program. (Typically, the web site is maintained by the 

these alternatives rely on spending more or less time in the proprietor that holds the copyright in the material, but this 

data analysis phases of watermark decoding, such as cross- need not be the case. Some material may be available at 

correlation operations. 35 several web sites, e.g., maintained by ABC Sports, the 

Reference has been made to watermarked UIDs as refer- National Basketball Association, and Sports Illustrated.) The 

ring to a database from which larger data strings (e.g. web search can use any of various known search engines, e.g., 

addresses, musician names, etc.) can be retrieved. In some Infoseek, Verity, etc., and can permit searching by title 

embodiments, the data record referenced by a UID can, in terms, keywords, date of airing, copyright owner, etc. By 

turn, point to several other database records. By such 40 typing in, e.g., the keyword Trailblazeis' and the date * Apr. 

arrangements, it is often possible to reduce the payload of 26, 1999,' the consumer is presented a listing of videos 

the watermark, since a single UID reference can lead to available for download. One, hopefully, is the requested 

several different data records. game. With each listing is an indication of an associated 

Production Tools nominal charge (e.g. 80 cents). 

In the prior art, the watermark embedded in a source 45 On clicking on a hypertext link associated with the 

material is typically consistent and static through a work — desired basketball game, the viewer is presented a further 

unchanging from beginning to end. But as will be reoog- screen with one or more options. The first of the hsted 

nized from the foregoing, there are many applications that options is the entire game, with commercials. The charge is 

are better served by changing the watermark data dynami- the nominal charge presented on the earlier screen (i.e. 80 

cally during the course of the work. According to another 50 cents). Other options may include the first, second, third, and 

aspect of the invention, a production tool is provided that fourth quarters of the game individually, each of which — 

facilitates the selection and embedding of dynamically- save the last, costs 20 cents. The last may be charged at a 

changing watermark data. One such embodiment is a soft- premium rate, e.g., 30 cents. Clicking on the desired video 

ware program having a user interface that graphically dis- option yields a further screen through which payment is 

plays the different watermark fields that are being embedded 55 effected. 

in a work, and presents a library of data (texdially or by To pay for the requested video, the consumer instructs his 

icons) that can be inserted into each field, and/or permits the or her computer to transfer three of the earlier-purchased 

user to type in data to be encoded. Another control on the UI tokens over the web to the video provider. Various user 

controls the advance and rewind of the media, permitting the interface metaphors can be employed to facilitate this 

user to determine the location at which different watermark 60 transfer, e.g., permitting the user to type the amount of 

data begins and ends. Graphical paradigms known from money to be transferred in a dialog box presented on-screen, 

video- and audio-editing tools can be used to indicate the or dropping/dragging icons representing tokens from an 

starting and ending frames/samples for each different water- on-screen "wallet" to an on-screen "ticket booth" (or over an 

mark payload. icon or thumbnail representing the desired content), clicking 

Such a tool can be of the standalone variety, or can be 65 on an "increment" counter displayed adjacent the listing of 

integrated into the desktop audio- and video- production and the content, etc. Once the consumer has authorized a transfer 

editing tools offered by vendors such as Avid, Adobe, Jaleo, of sufficient tokens, the consumer's computer sends to the 
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web site (or to such other web address as HTML encoding 
in the viewed web page may indicate) the tokens. This 
transmission simply takes the form of the three 128+ bit 
numbers (the *+' indicating the bank identifier) — in what- 
ever packet or other format may be used by the internet link. 5 
Once dispatched in this manner, the tokens are deleted from 
the user's computer, or simply marked as spent. (Of course, 
in other embodiments, a record of the expenditure may be 
stored in the consxuner's computer, e.g., with the token 
contents and a record of the audio or video purchase to 
which they were applied.) 

Since the amount of money is nominal, no encryption is 
provided in this embodiment, although encryption can natu- 
rally be provided in other embodiments (e.g., either in 
sending the tokens from the user to the web site, or earlier, 
in sending the tokens to the user). As will be seen, provided 
that the media provider immediately sends the tokens to the 
bank in real time, encryption is a nice feature but not 
mandatory. 

On receipt of the token data, the web site immediately 
routes the token data to the identified bank, together with an 20 
identifier of the media provider or account to which the 
funds represented thereby are to be credited. The bank 
checks whether the 128-bit numbers have been issued by 
that bank, and whether they have already been spent. If the 
numbers are valid, the bank updates its disk-based records to 25 
indicate that the three tokens have been spent and that the 
bank now owes the media supplier 30 cents, which it may 
either pay immediately (e.g., by crediting to an account 
identified by the media provider) or as one lump sum at the 
end of the month. The bank then sends a message to the web 30 
site confirming that the tokens were valid and credited to the 
requested account. (Optionally, a message can be sent to the 
purchaser of the tokens (if known), reporting that the tokens 
have been redeemed.) 

In response, the web site begins delivery of the requested 35 
video to the consumer. In the illustrated embodiment, the 
video is watermarked prior to delivery, but otherwise sent in 
unencrypted fashion, typically in streaming format, but 
optionally in file format. (Encryption can be used in other 
embodiments.) The watermarking in the illustrated embodi- 40 
ment is accomplished on-the-fly and can include various 
data, including the date of downloading, the download site, 
the destination IP address, the identity of the purchaser (if 
known), etc. 

The large size of the video and the small charge assessed 45 
therefore provide disincentives for the consumer making 
illicit copies. (Especially as to archival material whose value 
decays with time, there is not much after-market demand 
that could be served by illicit copies, making third party 
compilation of such material for re-distribution financially 50 
unattractive. First run video, and material that keeps a high 
value over time, would not be as well suited for such 
distribution, and could better employ technology disclosed 
elsewhere herein.) 

In some embodiments, the integrity of the received video 55 
is checked on receipt. This feature is described below in the 
section entitled Watermark-Based Receipts. 

In the illustrative system, nothing in the tokens indicates 
the identity of the purchaser. The web site knows the IP 
address of the site to which video was delivered, but need 60 
not otherwise know the identity of the purchaser. The bank 
would probably maintain a record of who purchased the 
tokens, but need not. In any event, such tokens could 
thereafter be exchanged among consumers, resulting in 
anonymity from the bank, if desired. 65 

As described above, the video excerpts from which the 
consumer can select include commercials. At some sites. 



video may be provided from which the commercials have 
been excised, or which is delivered in a manner that skips 
past the commercials without transmitting same to the 
consumer. Such video will naturally command a premium 
price. In some embodiments, the difference in price is 
electronically credited as compensation to accounts main- 
tained for (or by) the advertisers, whose advertisements are 
not being viewed by such consumers. (The identification of 
advertisers to be credited is desirably permanently encoded 
in the video, either throughout the video (if the video has had 
the commercials removed therefrom), or by data in the 
commercials themselves (which commercials are skipped 
for transmission to the consumer, but can still be decoded at 
the video head-end. Such encoding can be by in-band 
watermarking or otherwise.) 

While the foregoing discussion particularly considered 
video as the desired content, the same principles are equally 
applicable in connection with audio, still imagery, and other 
content. 

The token-based payment method is but one of many that 
can be employed; the literature relating to on-line payment 
mechanisms is extensive, and all such systems can generally 
be here-employed. 

Tracking 128-bit tokens can be a logistical problem for 
the bank. One approach is to have a memory with 10*^ 
locations, and at each location store a two-bit value (e.g. 
00-never issued; 01 -issued but not spent; 10-issued and 
spent; 11 -reserved). More complete data could alternatively 
be stored, but such a memory would be impractically large. 

One alternative approach is to hash each 128-bit number, 
when issued, to a much smaller key value (e.g. 20 bits). A 
memory with 10^° locations can be indexed by this key. 
Each such location can include four data: an issued 128-bit 
token number that hashes to that value, first and second date 
fields indicating the date/time on which that token was 
issued and redeemed, respectively, and a link specifying the 
address of a next memory location. That next memory 
location (outside of the original 10^ locations) can include 
four more data, this time for a second issued-1 28-bit token 
number that hashed to the original key value, two date fields, 
and again with a link to a subsequent storage location, etc. 

When a 128-bit random number is generated, the original 
memory location indexed by the hash code of that number 
is checked for an earlier number of the identical value (to 
avoid issuance of duplicate tokens). Each successive loca- 
tion in the linked chain of memory locations is checked for 
the same 128-bit number. When the end of the linked chain 
is reached, the bank knows that the 128-bit random number 
has not previously been issued, and writes that number in the 
last-addressed location, together with the date of issuance, 
and a link to a next storage location. 

When a 128-bit token is received, the same linked-list 
processing occurs to identify a first location, and to there- 
after step through each subsequent location until a match is 
found between the token number and the number stored in 
one of the linked memory locations. When found, that 
nimiber is marked as redeemed by writing a redemption 
date/time in the corresponding field. If the search reaches the 
end of the linked chain without finding a match between the 
stored niunbers and the token number, the token is treated as 
invalid (i.e. not issued by that bank). 

Other manners of tracking the large number of possible 
token numbers can of course be used; the foregoing is just 
exemplary. Or the tokens needn't be tracked at all. Such an 
arrangement is highly practical if the token has sufficient 
bits. With the illustrated 128 bits, for example, the chance of 
two identical tokens being issued is infinitesimally small, so 
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checking for duplicate issuance can be omitted if desired In 
such case, the bank can simply maintain an ordered list of 
the token numbers still outstanding and valid. As new tokens 
are dispensed, their token numbers are added to the list. As 
tokens are redeemed, their numbers are deleted from the list. 5 
Known list processing techniques can be employed to speed 
such search, update, and delete actions. 
Watermark-Based Receipts 

Pay-for-content applications commonly assume that if 
content is transmitted from a server (or head-end, etc), it is 10 
necessarily received. Sometimes this assumption is wrong. 
Network outages and interruptions and internet traffic load 
can diminish (e.g., dropped video frames), or even negate 
(e.g., failed delivery), expected consumer enjoyment of 
content. In such cases, the consumer is left to haggle with the 15 
content provider in order to obtain an adjustment, or refund, 
of assessed charges. 

Watermarks provide a mechanism for confirming receipt 
of content. If a watermark is detected continuously during a 
download or other delivery event, a software program (or 20 
hardware device) can issue an electronic receipt attesting 
that the content was properly delivered. This receipt can be 
stored, and/or sent to the content distributor to confirm 
delivery. 

In one embodiment, a content receiving device (e.g., 25 
computer, television or set-top box, audio appliance, etc.) 
periodically decodes a watermark from the received content 
to confirm its continued reception. For example, every five 
seconds a watermark detector can decode the watermark and 
make a record of the decoded data (or simply record the fact 30 
of continued detection of the same watermark). When a 
changed watermark is detected (i.e., reception of a different 
content object begins), the duration of the previously- 
received content is logged, and a receipt is issued. 

In a related embodiment, the last portion (e.g., 5 seconds, 35 
frame, etc.) of the content bears a different "end of content" 
watermark that triggers issuance of a receipt. Such a water- 
mark can indicate the length of the content, to serve as a 
cross-check against the periodic watermark polling. (E.g., if 
periodic sampling at 2 second intervals yields 545 samples 40 
corresponding to the same content, and if the "end of 
content" watermark indicates that the content was 1090 
seconds long, then receipt of the entire content can be 
confirmed.) 

In another embodiment, the watermark can change during 45 
the course of the content by including, e.g., a datum that 
increments every frame or other increment of time (e.g., 
frame number, time stamp, etc). A watermark detector can 
monitor the continued incrementing of this datum through- 
out the content to confirm that no part was garbled (which so 
would destroy the watermark) or was otherwise missing. 
Again, at the end of delivery, the receiving system can issue 
a confirmation that XXX frames/seconds/etc of the identi- 
fied content were received. 

One application of such technology is to bill for content 55 
based on receipt, rather than transmission. Moreover, bill- 
ings can be adjusted based on percentage of content-value 
received. If delivery is inlermpted mid- way through (e.g., by 
the consumer disabling the content-receiving device), the 
nominal billing for the content can be halved. Some pro- 60 
longed content, e.g., televised/web-broadcast university 
classes, cannot be "consimied" in one session, and are thus 
particularly well suited for such pay-as-you-consume bill- 
ing. 

Another application of such technology is in advertising 65 
verification. Presently, ads are tracked by transmission or, 
less frequently, by detection of an embedded code on receipt 
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(cf., Nielsen Media Research's U.S. Pat. Nos. 5,850,249 and 
5,737,025). However, such reception-detectors — once 
triggered — generally do not further note the length of time 
that the advertising was received, so the same data is 
produced regardless of whether only five or fifty seconds of 
a commercial is presented. Watermark monitoring as con- 
templated herein allows the duration of the advertising 
impression to be precisely tracked. 

In one application of this technology, recipients of adver- 
tising are provided incentives for viewing advertising in its 
entirely. For example, a content-receiving device can 
include a watermark detector that issues a receipt for each 
advertisement that is heard/viewed in its entirety. These 
receipts can be redeemed, e.g., for content tokens as 
described elsewhere herein, for monetary value, etc In some 
embodiments, receipts are generic and can all be applied to 
a desired premium, regardless of the advertisements through 
which they were earned. In other embodiments, the receipts 
are associated with the particular advertisers (or class of 
advertisers). Thus, a TV viewer who accumulates 50 receipts 
from advertising originating from Procter & Gamble may be 
able to redeem same for a coupon good for $2.50 off any 
Procter & Gamble product, or receipts from Delta Airlines 
may be redeemed for Delta frequency flier miles (e.g., at a 
rate of one mile per minute of advertising). Such incentives 
are particularly useful in new forms of media that give the 
consumer enhanced opportunities to fast-forward or other- 
wise skip advertising. 

(Although the foregoing "receipt" concept has been 
described in conjunction with watermark data (and use of 
watermark technology is believed to be inherently advanta- 
geous in this application), the same principles can likewise 
be implemented with ancillary data conveyed by other 
means.) 

Master Global Address 

As suggested above, it is desirable that each piece of 
content have a web address (the "Master Global Address" 
(MGA), or "Master IP Address") associated with it. Such 
address is typically conveyed with the content, e.g., by an IP 
address watermarked therein. 

Consider a consumer who downloads a streaming video 
having an English language soundtrack. The viewer may not 
speak English, or may otherwise prefer to listen to the 
soundtrack in another language. The user can decode the 
watermark data embedded in the video and initiate a link to 
the associated web address. There the user is presented with 
a list of soundtracks for that content object in other lan- 
guages. The viewer can click on the desired language and 
receive same via a second simultaneous transmission (e.g., 
a second socket channel). The consumer's audioMdeo 
appliance can substitute the desired audio track for the 
default English track. 

If the streaming video and the alternative soundtrack are 
hosted on the same server, synchronization is straightfor- 
ward. The process governing transmission of the alternative 
soundtrack identifies the process that is streaming video to 
the same IP address. Based on SMPTE, or other time frame 
data, the former process syncs to the latter. (If the two data 
streams don't originate through the same server, time/frame 
data can be relayed as necessary to the alternative 
soundtrack server to effect synchronization.) 

Another application of the Master Global Address is to 
serve as a point to which monitoring stations can report the 
presence, or passage, of content. Consider, for example, a 
copyright-aware node through which content signals pass, 
e.g., a computer node on a network, a satellite transponder, 
etc. Whenever the node detects passage of a media object 
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(e.g., by reference to a file extension, such as MP3, JPG, Clicking on option (d) proceeds as above, and permits full 
AVI, etc.), it sends a "ping*' over the internet to the address use of the image on the computer. Moreover, the MGAsite 
encoded in the object, simply reporting passage of the sends a digital certificate to the user's computer memorial- 
object. Similar monitoring facilities can be provided in end izing the usage rights purchased by the consumer, 
user computers, e.g., reporting FileOpen, FileSave, Printing, 5 In this particular arrangement, no access control is placed 
or other use of content bearing MCA data. on the content, e.g., by encryption, secure container 
This system can be expanded to include "ping" and technology, or the like. The nominal fees, and the ease of 
"pong" phases of operation. When a software application (or licensing, make it simple for the user to "do the right thing" 
a user appliance, such as a video or audio playback device) and avoid copyright liability. In other embodiments, of 
encoimters a media object (e.g., at time of file open, at time lO course, known access control techniques can be used to limit 
of playback, etc.), it pings the MGA site to report the use of the object until the requisite payment has been made, 
encounter. The MGA site "pongs" back, responding with Naturally, records of all such transactions are also logged 
instructions appropriate to the encounter. For example, if the at the MGA site. 

object requires payment of a fee before full functionality or Clicking on option (e) opens a browser window on the 

access is to be granted, the MGA site can respond to the 15 user's computer to a web site that presents a complete listing 

application with instructions that the object be used (e.g., of license options available for that image. (The address of 

played back) only in some crippled state preventing the this web site is included in customization data relayed to the 

user's fill! enjoyment (e.g., impaired resolution, or impaired user device firom the MGA site, but not explicitly shown to 

sound quality, or excerpts only, etc.). The MGAsite can also the user on the menu.) Through such web site, the user can 

inform the user application of the terms (e.g., payment) by 20 select desired rights, effect payment, and receive the neces- 

which full functionality can be obtained. The application can sary authorization for software applications on the user's 

graphically or audibly present such information to the user, computer (or other media appliance) to open and/or process 

who can authorize a payment, if desired, so that the content the content. 

can be enjoyed in a less- (or im-) crippled state. On receipt The object on which the user "clicks" needn't be an icon, 

of the payment authorization, the MGA site can inform the 25 It can be an image or other graphical representation. (And a 

user application that enhanced accessAisage rights have been "click" isn't necessary; a voice command or other signal 

purchased, and that the application may proceed accord- may be used to the same effect with an audio clip or 

ingly. selection.) 

Yet another application of the MGA is to present the user Consider the popular merchandising of books and CDs 

of a content object a menu of options that is customized to 30 over the internet. A JPG or other image file depicting the 

that object. cover of a book, or the artwork of a CD cover, can be treated 

In current graphical operating systems, when a user clicks as a media object, and can include a watermarked MGA 
on an icon (e.g., with the right mouse button), a menu is pointer. Right-clicking on such an image of a book cover 
presented detailing actions that can be undertaken in con- could, through the MGA site, present to the user a menu of 
nection with the icon, or the file represented thereby. Such 35 options that includes — in addition to those normally pre- 
options are pre-programmed (i.e., static), and are typically sented in conjunction with a JPG file — the following: 
determined by the operating system based solely on the file (a) "See the review of this book published in the New 
extension. York Times on Apr. 19, 1999" 

In accordance with this aspect of the present invention, (5) "3^^ the list of reviews of this book at Amazon.com" 

clicking on an icon representing a media object initiates an 40 . . ^^^.^^ ^^^^^ p^^tj^g 

internet link to the MGA site associated with the object. The Amazon com" 

MGA ate responds with data that is used to customize the ^ ^^^^ ^^ Amazon.com" 

menu of options presented to the user m connection With that /xuTvi. i.-uir a r f^^r^c** 

^. 1 . . / (e) Purchase this book from Amazon.com for $16.95 

particular object. ^ 

Consider an icon representing a JPG image file. Right- 45 (0 "Purchase this book from Barnesandnoble.com for 

cUcking on the icon may yield a menu that gives the user $19.95 and receive a $5.00 credit towards your next 

various options presented by the operating system (e.g., purchase 

delete, compress, rename), and additional options custom- (g) "Link to the web site that tells about the release of this 

ized in accordance with data from the object's MGA site. title as a motion picture (presently scheduled to open on 

These customized options may include, e.g., 50 Oct. 10, 1999)" 

(a) open in 100x150 pixel format for free; C^k"^ ^^"^^ ^^^""^ ''^^^ ""^'^'"^ 

(b) open in 480x640 pixel format for ten cents; ,it^ . t ^ * - 1 • i_- t 1 » 

^ „ (1) Search Lycos for ustings relating to this book, 

(c) open in 960x1280 pixel format for twenty cents; ^^^^ purchase options from the 

(d) purchase rights to use this image in a newsletter 55 menu, a pre-stored e-commerce profile — containing the user 
having a circulation of under 1000 for $1.25; name, credit card number, billing address, ship-to address, 

(e) display a complete listing of license options. etc., possibly in the form of an encrypted object — could be 
Clicking on options (b) or (c) initiates a commerce sent to the MGA site (or to the bookseller) to effect the 

application through which funds are electronically trans- purchase, or such selection could initiate display of addi- 

ferred to the MGA site (by the above-described tokens or 60 tional screens or sub-menus through which the user would 

otherwise). In response, the MGA site responds (e.g., with manually enter or select such information for transmission. 

TCP/IP or HTML instructions) authorizing an application on Others of the selections cause a new browser window to 

the user's computer to open the file in the requested manner. open on the user's computer, opening to a URL specified in 

(The default application for JPG applications can then data relayed from the MGAsite but not displayed to the user 

automatically be launched, or the computer may first query 65 in the menu. Appropriate HTML instructions can be gener- 

the user whether another application should be used ated to effect a particular query or other operation at the 

instead.) specified URL. 
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Iq some embodiments, the customized menu presents 
only a single choice in addition to those normally provided 
by the operating system, e.g., "Link to home." Clicking on 
this option opens a browser window to a home page at the 
MGA for that object. On that page, the user is presented with 5 
all of the foregoing options, and more (possibly including 
advertising graphics or multi-media). Such objects can serve 
as powerful marketing agents. Returning to the example 
discussed above, a JPG image file of a book cover may have, 
as its MGA, a web page hosted by a particular bookseller, 10 
providing purchase options and other information for that 
book. Marketing of books (or CDs, or cars, or consumer 
appliances, or virtually anything else) can be effected by 
disseminating such vendor-issued JPGs as widely as pos- 
sible. Some book cover JPGs may be distributed by 15 
Amazon.com, others by Barnes&Noble.com, others by 
Borders.com — each pointing back to a different MGA 
through which purchase transactions for that book may be 
performed. 

Returning to the MGA-customized menus, these needn't 20 
be limited to menus resulting from clicking on an icon or 
image (or signaling during an audio excerpt). Drop-down 
menus in application programs can likewise be populated 
with customized options, in accordance with customization 
data obtained from the MGA site for the object presently 25 
being accessed or used. Most graphical operating systems 
and application programs have well developed toolsets 
permitting such menu customization. Again, other data 
relayed from the MGA site is not shown to the user, but is 
employed by the computer (e.g., a browser program) to carry 30 
out menu options selected by the user. 

Again the foregoing techniques are equally applicable for 
still images, audio, video, and other forms of content, and 
can reachly be adapted for use both with general purpose 
computers, software applications, and specialized media 35 
appliances. 

While, for expository convenience, the foregoing discus- 
sion contemplated embedding a literal URL address in the 
object as the MGA, more typically this is not the case. 
Instead, the MGA more commonly comprises identification 40 
data for the object (e.g. a 128-bit random ID), together with 
the URL for a name server computer that serves many 
(perhaps millions) of such objects (an example of the latter 
is the Digimarc MarcCentre server). 

To obtain the desired data as detailed above, the user's 45 
computer (sometimes termed a client computer) links to the 
name server computer and provides the ID of the object 
being processed. The name server computer uses this ID to 
query a database, and obtains from the database the current 
IP address to which such queries should be routed. The name so 
server computer can relay the request from the client com- 
puter to the correct destination address, or can return the 
correct destination address to the client computer, which can 
initiate such a link itself. By such arrangement, the IP 
address ultimately associated with an object can be easily ss 
changed as needed, simply by changing the corresponding 
record in the name server database, without rendering obso- 
lete legacy objects having out-of-date addresses encoded 
therein. 

In some embodiments, the URL of the name server 60 
needn't be included in the watermark In the absence of a 
specified URL, the client computer may direct such links to 
a default name server address instead (stored locally or 
remotely). If that server doesn't recognize the object ID, it 
can return an error code, or pass the query on to other name 65 
servers. Those servers, in turn, can pass the query along to 
still other name servers if they don't recognize the object ID. 
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In this fashion, an exponentially-large number of name 
servers might be quickly polled for information relating to 
the identified object. Alternatively, rather than encoding the 
complete IP address of the name server in an object 
watermark, the first N (e.g., 16) bits of the object ID might 
be xised as a short-hand for one of 65,536 predetermined 
name server addresses, in accordance with data stored 
locally (e.g., on RAM or disk in the user's computer) or 
remotely (e.g., at a default name server IP address). 

While the basic concept idea behind embedding MGA 
data within an object is to point to a repository of data about 
the object, a pointer the other way may be achieved as well. 

As noted, the "ping" application of MGA data permits an 
MGA site to be informed of sites through which its object 
passes. More generally, the MGA site can log the originating 
address of each query it receives. Each such address can be 
presumed to have (or have had) a copy of the corresponding 
object. Media owners can thereby track the dissemination of 
copies of their media objects — at least insofar as use of such 
objects entails communicating with the associated MGA 
site. 

Such tracking offers a great number of opportunities, 
some in the area of commerce. The MGA site corresponding 
to the cover art of a Garth Brooks CD, for example, can 
provide a listing of IP addresses of persons interested in that 
CD. Email or promotional data objects (e.g., audio clips) can 
be sent to that list of addresses when a subsequent Garth 
Brooks CD is released. 

Such tracking also opens up a new dimension of internet 
searching. Presently, internet search engines use a brute 
force approach, visiting millions of pages across the web in 
order to identify, for example, a dozen instances of a given 
photograph file. MGAs offer a shortcut to such brute force 
approaches. With the present technology, a search engine 
can find a single instance of a photograph file and, by 
detection of the MGA data watermarked therein, link to the 
corresponding MGA site. From the MGA site, the search 
engine can obtain a listing (if such queries are authorized) of 
some or all of the other sites known by the MGA site to have 
copies of that photograph file. (Providing such data to search 
engines is a commerce opportunity for such MGA sites, 
which may permit such access to its listing of sites only in 
exchange for a fee. Or the MGA site may arrange to collect 
a tribute payment from the search engine proprietor each 
time the engine responds to a user query using data collected 
from the MGA site.) 

Many of the addresses logged by the MGA may not be 
publicly-accessible data stores. The search engine can check 
each listed address to ensure that the desired object is present 
and accessible before adding the address to its database. 

Covert Tracing 

Co-pending application Sen No. 09/185380 describes 
anti-counterfeiting technology that looks for the presence of 
digital data corresponding to bank note imagery in a com- 
puter system, and makes a covert record of any attempt to 
process such data (e.g.. Scan, FileOpen, FileSave, Print, 
Edit, etc.). Such records are hidden from the user of the 
system (using, e.g., various data encryption and obscuring 
techniques), but authorized law enforcement officials are 
provided tools by which these records can be recovered. The 
forensic data thereby obtained may prove useful in pros- 
ecuting counterfeiters. (Knowledge that a computer may be 
covertly storing evidence of attempted counterfeiting 
actions may prove as, or more, valuable in deterring coun- 
terfeiting; than the covert records themselves.) 

The same techniques can be employed to deter unautho- 
rized processing of audio, image, video, or content by media 
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pirates. In one embodiment, a computer's operating system 
(including peripheral device drivers) monitors various data 
within the system (e.g., data sent to writeable storage media, 
or sent via a serial port or network connection, etc.) for data 
bearing a do-not-copy watermark The presence of such data 5 
being sent, e.g., to a writeable disk or to a remote computer, 
indicates that the do-not-copy instruction has been circum- 
vented. In such case, the operating system writes one or 
more covert records memorializing the activity, for possible 
use in criminal prosecution if the computer is lawfully 
seized. 

The example just-provided is but one of many monitoring 
and response techniques that may be employed to deter 
circumvention of copy-protection or other access control 
systems. Generally speaking, if content data is foimd where 
it shouldn't be, or is found used as it shouldn't be used, a 
corresponding record should be made. (Other intervention 
actions can be triggered as well; covert tracing is desirably 
just one of several parallel responses to suspected hacking.) 

Meta-Data Accessed Using Watermarks 

Meta-data, in formats known as XML, SGML, and 20 
HTML, is widely used to communicate information about 
digital objects (e.g., author, keywords, price, rights, caption, 
etc.). More generally, meta-data can be thought of as any 
data construct which associates the name of a property (e.g., 
"author), with the value of the property (e.g., "Mark 25 
Twain'*). Such data commonly appears in a tag format, such 
as the following: 

<META NAME-"author" CX)NTENT-"Mark Twam"> 

Meta-data is commonly exchanged between server and 3Q 
client computers in conjunction with the digital objects to 
which they relate (e.g., the text of a Mark Twain book). 

As detailed herein, an important application of water- 
marking is likewise to convey information about media — ^in 
this case embedded within the media content itself (e.g., 
providing unique identification, establishing some basic 
behaviors such as do not copy, and providing links to 
extended functionality). 

For meta-data to be useful, it must be linked to associated 
content, whether in the context of a browser, application 
program, operating system, asset management system, 40 
search engine, etc. However, as detailed below, the content 
and the associated meta-tags needn't always be conveyed 
together. 

Consider an apphcation program or other client process 
that receives a watermarked media object. The watermark 45 
includes an MGA for that object (which, as noted above, 
may not specify an ultimate IP address). Stored at the MGA 
site is meta-data corresponding to the object. By linking to 
the MGA site identified by the object's watermark, the client 
computer can obtain the meta-data corresponding to the 50 
object. This data can be stored at the client computer and 
used just as any other meta-data, e.g., to define the local 
functions that should be available for use with that object 
(e.g., buy, search, etc.). 

A particular example is an on-line catalog of stock pho- 55 
tography. Each photograph is watermarked with MGA data. 
To identify the photographer, copyright date, price, tele- 
phone number, subject, etc., an application program can link 
to the MGA site for that photograph, and obtain the corre- 
sponding meta-data. This data can then be displayed or used 60 
as needed. Data objects of disparate formats thus can readily 
be handled within a single, simple application program, 
since the program needn't concern itself with the varying 
formats for the associated meta-data (assuming the name 
servers provide this data in standardized format). Substantial 65 
flexibility in programming and object formatting is thereby 
achieved. 
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Returning to the internet search engine example described 
above, MGAs may become recognized as repositories rich 
in meta-data for media objects. Specialized search engines 
may focus their data collection around such sites, and be 
able to quickly identify the MGA sites corresponding to 
various boolean combinations of meta-tag parameters. 

Asset Management/Containers 

Much has been written on the topic of asset rights 
management. Sample patent documents include U.S. Pat. 
Nos. 5^92,900, 5,715,403, 5,638,443, 5,634,012, 5,629,980 
and laid-open European application EP 862,318. Much of 
the technical work is memorialized in journal articles, which 
can be identified by searching for relevant company names 
and trademarks such as IBM's Cryptolope system, Portland 
Software's ZipLock system, the Rights Exchange service by 
Softbank Net Solutions, and the DigiBox system from 
InterTrust Technologies. 

An exemplary asset management system makes content 
available (e.g. from a web server, or on a new computer's 
hard disk) in encrypted form. Associated with the encrypted 
content is data identifying the content (e.g. a preview) and 
data specifying various rights associated with the content. If 
a user wants to make fuller use of the content, the user 
provides a charge authorization (e.g. a credit card) to the 
distributor, who then provides a decryption key, allowing 
access to the content. (Such systems are often realized using 
object-based technology. In such systems, the content is 
commonly said to be distributed in a "secure container.") 

Desirably, the content should be marked personalized/ 
seriahzed) so that any illicit use of the content (after 
decryption) can be tracked. This marking can be performed 
with watermarking, which assures that the mark travels with 
the content wherever — and in whatever form — it may go. 
The watermarking can be effected by the distributor — prior 
to dissemination of the encrypted object — such as by encod- 
ing a UID that is associated in a database with that particular 
container. When access rights are granted to that container, 
the database record can be updated to reflect the purchaser, 
the purchase date, the rights granted, etc. An alternative is to 
include a watermark encoder in the software tool used to 
access (e.g. decrypt) the content. Such an encoder can 
embed watermark data in the content as it is released from 
the secure container, before it is provided to the user. The 
embedded data can include a UID. This UID can be assigned 
by the distributor prior to disseminating the container. 
Alternatively, the UID can be a data string not known or 
created until access rights have been granted. In addition to 
the UID, the watermark can include other data not known to 
the distributor, e.g. information specific to the time(s) and 
manner(s) of accessing the content. 

As noted earlier, access rights systems can be realized 
with watermarks without containers etc. For example, in a 
trusting world, copyrighted works can be freely available on 
the web. If a user wishes to make lawful use of the work, the 
user can decode its watermark to determine the work's terms 
and conditions of use. This may entail linking to a web site 
specified by the embedded watermark (directly, or through 
an intermediate database), which specifies the desired infor- 
mation. The tiser can then arrange the necessary payment, 
and use the item knowing that the necessary rights have been 
seciu'ed. 

Remote Reconfiguration of Watermark Detectors 
In some cases, it is desirable to reconfigure watermark 
detectors remotely. Such functionality is desirable, for 
example, if a watermark system is hacked or otherwise 
compromised. 

In accordance with this aspect of the present invention, 
some aspect of a watermark detector's operation is changed 
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in response to a command. The change can take various 
forms. In watermark systems employing pseudo-random key 
data (e.g., spread spectrum spreading signals), the pseudo- 
random signal used for detection can be changed. In systems 
using DFT processing, the mapping between message bits 5 
and DFT coeflScients can be changed. In still other systems, 
the decoding can proceed as before, but the significance of 
one or more bits can be changed (e.g., bits that were 
normally interpreted as defining Field A can be interpreted 
as defining Field B, and vice versa). In yet other systems, the 
decoding can proceed as before, but the response of a device 
to a given watermark signal can be changed. In still other 
systems, a set of software instructions can be re-written or 
re-ordered to effect a change in detector operation. 

The command can be conveyed in various ways. In one 
embodiment, it can be a trigger bit in the watermark payload. 
Normally the bit has a value of "0." If the bit has a value of 
"1," the detector system responds by changing its operation. 
A trigger pattern can also be established, so that detection of 
a certain combination of bits in the watermark payload 
serves to trigger the change. Reserved states of certain data 20 
fields are examples of patterns that might be employed. 

The command can also be conveyed through another 
channel different than the watermark channel (e.g., an SCA 
channel of an FM broadcast, or the sub-titling data channel 
of video broadcasts, or header data within an MPEG data 25 
stream, etc., etc.). 

The change can proceed in accordance with a pre- 
programmed rule (e.g., codes progressing successively 
through a numerically or algorithmically-determined 
progression), or the change can proceed in accordance with 30 
data specified elsewhere in the payload of the watermark 
bearing the trigger bit (e.g., instead of being interpreted in 
normal fashion, the non-trigger bits of the detected water- 
mark can define a new pseudo-random key data. Or the 
change can proceed in accordance with data conveyed in 35 
successively-presented watermark payloads, as might be 
done in video encoding where each frame of video can 
convey further watermark information. (This latter arrange- 
ment is one offering a high-bandwidth re-programming 
channel through which, e.g., extensive firmware instructions 40 
might be transferred to the detector to replace instructions 
earlier stored.) 

By such arrangements, greatly increased detector versa- 
tility and functionality can be achieved. 

Conclusion 45 

Many diverse embodiments are reviewed above — each 
with a unique set of features. (Still others are disclosed in the 
assignee's patents incorporated by reference.) This specifi- 
cation should be construed as explicitly teaching that fea- 
tures illustrated in one such embodiment can generally be 50 
used in other embodiments as well. Thus, for example, a date 
field was not particularly discussed in connection with 
payload data for video watermarking. Nor were "play once" 
watermarks so-considered. The inclusion of a calibration 
signal with (or as part of) the watermark is shown in 55 
embodiments of the issued patents, but is not belabored in 
the above-described embodiments. Likewise with "simple 
universal codes." The pre-stored commerce profile 
described in one of the foregoing embodiments is equally 
applicable to other embodiments as well. Likewise, the 60 
presentation of advertising was discussed in connection with 
one embodiment but not others, although it, too, is generally 
applicable. All of these concepts are familiar at Digimarc 
and are regarded as generally applicable throughout the 
work expressed in Digimarc *s patent disclosures. Practical- 65 
ity prevents an exhaustive recitation of each individual 
permutation and combination. 



,214 Bl 

66 

Having described and illustrated the principles of our 
invention with reference to illustrative embodiments, it will 
be apparent that the detailed arrangements can be modified 
in arrangement and detail without departing from such 
principles. 

For example, while reference has been made to various 
uses of wireless, it should be understood that such reference 
does not just cover FM broadcast, and wireless internet 
networking and the like, but also includes other wireless 
mechanisms. Examples include cell phones and direct sat- 
ellite broadcast. 

Likewise, while certain embodiments were illustrated 
with a watermark payload of 100+ bits, in other systems 
much smaller (or sometimes larger) payloads are 
desirable — sometimes as small as 1-8 bits. 

While the foregoing examples have each been illustrated 
with reference to a particular media type (e.g., video, audio, 
etc.), it will be recognized that the principles of each 
embodiment find application with the other media types as 
well. 

Certain of the appliances contemplated above reqiiire user 
interfaces more sophisticated than are presently typical on 
such devices. The simplicity of the underlying audio appli- 
ance can be preserved, in many instances, by using a 
palmtop computer — coupled by infrared or otherwise — as a 
temporary user interface to the appliance. Some of the 
processing capabifity can likewise be off-loaded to an ancil- 
lary palmtop. (Palmtop is here meant to refer generally to 
any pocket-size programmable computing device.) Unless 
otherwise stated, it should be understood that the digital 
music, video, and imagery contemplated herein is not of any 
particular form or format. Audio, for example, can be of 
various forms, both streaming and non-streaming, and of 
various formats (e.g. MP3, MP4, MS Audio, Windows 
Media Technologies, RealAudio, *.WAV, MIDI, Csound, 
Dolby's Advanced Audio Codec (AAC), etc. 

To provide a comprehensive (Hsclosure without unduly 
lengthening the present specification, applicants incorporate 
by reference the patent publications and applications cited 
herein. 

We claim as our invention all such embodiments as may 
come within the scope and spirit of the following claims, and 
equivalents thereto: 

1. A method comprising: 

presenting a digitally encoded object to an optical sensor, 
the optical sensor producing output data; 

decoding plural-bit data from the sensor output data; and 

using said plural-bit data to establish a link to an internet 
address having data relating to said object; 

wherein the object is steganographically encoded with 
said plural-bit data. 

2. The method of claim 1 in which the object is a business 
card. 

3. The method of claim 1 in which the object comprises 
printed advertising, 

4. The method of claim 1 in which the object comprises 
product packaging. 

5. The method of claim 4 in which the product packaging 
comprises a cover associated with packaged music media. 

6. The method of claim 1 in which the object comprises 
a portion of a book. 

7. The method of claim 1 in which the object comprises 
an article of postal mail. 

8. The method of claim 1 in which the object comprises 
printed advertising. 

9. The method of claim 1 in which the plural-bit data 
comprises a code, the method including consulting a data 
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Structure to obtain an internet address associated with the 
code, and initiating a link to said address. 

10. A method of initiating access to a computer via a data 
communications medium, the method comprising: 

receiving data corresponding to an object, said object 
comprising visual data and having information indica- 
tive of an address associated with the computer stega- 
nographically embedded in-band within said visual 
data; 

decoding the information from said object; and 
initiating a link to the computer using said information; 
said decoding and initiating being performed by the same 
device. 

11. The method of claim 10 in which said receiving 
comprises receiving said object from a digital storage or 
transmission medium in digital form, without said object 
having being rendered in human-perceptible form since 
being steganographically embedded. 

12. The method of claim 10 in which said receiving 
comprises sensing a human-perceptible form of said object, 
as by an optical sensor device. 

13. A method of initiating access to a computer via a data 
communications medium, the method comprising: 

receiving data corresponding to an object, said object 
comprising visual data and having information indica- 
tive of an address associated with the computer stega- 
nographically embedded in-band within said visual 
data; 

decoding the information from said object; and 
initiating a link to the computer using said information; 
wherein said receiving comprises receiving said object 
from a digital storage or transmission medium in digital 
form, without said object having being rendered in 
human-perceptible form since being steganographi- 
cally embedded. 

14. A method of initiating access to a computer via a data 
communications medium, the method comprising: 

receiving data corresponding to an object, said object 
comprising visual data and having information indica- 
tive of an address associated with the computer stega- 
nographically embedded in-band within said visual 
data; 



decoding the information from said object; and 
initiating a link to the computer using said information; 
wherein said visual data includes plural samples, said 
steganographically embedded information extending 
generally throughout said samples, rather than local- 
ized in one or more particular portions thereof, wherein 
the information can be decoded from an excerpt of said 
visual object and used to initiate the link to the com- 
puter. 

15. A method of initiating access to a computer via a data 
conununications medium, the method comprising: 

receiving image data corresponding to a printed object, 
the printed object including both text and backgroimd, 
at least the background having information indicative 
of an address associated with the computer stegano- 
graphically embedded therein; 
decoding the information from said image data; and 
initiating a link to the computer using said information. 

16. The method of claim 15 in which said information 
comprises a code, the method including consulting a data- 
base to obtain a computer address associated with said code, 
and initiating a link to said address. 

17. The method of claim 15 in which the printed object 
comprises a business card. 

18. The method of claim 15 in which the printed object 
comprises printed advertising. 

19. The method of claim 15 in which the printed object 
comprises product packaging. 

20. The method of claim 15 in which the product pack- 
aging comprises a cover associated with packaged music 
media. 

21. The method of claim 15 in which the printed object 
comprises a portion of a book. 

22. The method of claim 15 in which the printed object 
comprises an article of postal mail. 

23. The method of claim 15 in which the printed object 
40 comprises printed advertising. 
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UNITED STATES PATENT AND TRADEMARK OFFICE 

CERTIFICATE OF CORRECTION 



PATENT NO. : 6,311,214 Bl Page 1 of 1 

DATED : October 30, 2001 

INVENTOR(S) : Geoffrey B. Rhoads 

It is certified that error appears in the above-Identified patent and that said Letters Patent is 
hereby corrected as shown below: 



Title page. 

Item [63], change "which is" to -- and --, after "Apr 15, 1999" add -- which claims 
priority to application 60/082,228, filed on April 16, 1998 --. 

Colurmi 7, 

Line 35, change "data The" to data. The --. 
Column 19, 

Line 34, change "comers" to - corners --. 
Column 52, 

Line 38, change "samples pixels)" to ~ samples (pixels) 
Column 57, 

Line 1, change "desired In" to ~ desired. In 
Column 58, 

Line 11, change "entirely" to - entirety -. 
Column 61, 

Line 61, change "watermark In the" to - watermark. In the --. 
Column 63, 

Line 5, change "watermark The presence" to watermark. The presence -. 



Signed and Sealed this 
Ninth Day of September, 2003 




JAMES E. ROGAN 
Director of the United States Patent and Trademark Office 
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